Jamesbond — Paper 2 per-game results

Every method's faithfulness on Jamesbond

Each XAI / mechanistic-interpretability method's faithfulness on this ROM, sorted by F (click a header to re-sort). The left border marks the tradition: green = causal / intervention, red = gradient / correlational, amber = other — the split the paper's headline turns on. F is all-regime; content-F and position-F are the two output regimes (n/a where this game has no record for that regime).

Method	Family	F (all)	content-F	position-F	S	M
A8 · Whole-state recording	descriptive	1.000	n/a	n/a	1.000	0.055
ACDC — automatic circuit discovery	causal / intervention	1.000	n/a	n/a	0.385	1.000
Activation patching / causal mediation	causal / intervention	1.000	n/a	1.000	1.000	0.909
Causal scrubbing	causal / intervention	1.000	n/a	n/a	1.000	0.500
Interchange interventions / DAS	causal / intervention	1.000	n/a	n/a	1.000	0.769
A2 · Single-unit lesions	causal / intervention	0.985	n/a	n/a	0.600	1.000
Logit / tuned lens	causal / intervention	0.764	n/a	n/a	n/a	0.105
A5 · Local field potentials	gradient / correlational	0.743	n/a	n/a	n/a	n/a
Occlusion	causal / intervention	0.724	0.747	0.700	0.893	0.964
KernelSHAP / Shapley	gradient / correlational	0.719	0.747	0.691	0.893	0.288
LIME	gradient / correlational	0.684	0.727	0.640	0.893	0.288
RISE	gradient / correlational	0.644	0.685	0.603	0.893	0.288
A7 · Dimensionality reduction (NMF/PCA)	dim_reduction	0.600	n/a	n/a	n/a	n/a
Sparse autoencoders	dim_reduction	0.580	n/a	n/a	-0.097	0.875
On-distribution counterfactual	causal / intervention	0.578	0.747	0.408	0.893	1.000
Grad×Input / DeepLIFT	gradient / correlational	0.447	0.865	0.028	0.893	1.000
Extremal / meaningful perturbation	causal / intervention	0.402	0.489	0.314	0.893	0.964
Expected Gradients	gradient / correlational	0.388	0.747	0.028	0.893	1.000
Integrated Gradients	gradient / correlational	0.388	0.747	0.028	0.893	1.000
Vanilla gradient (saliency)	gradient / correlational	0.388	0.747	0.028	0.893	1.000
SmoothGrad	gradient / correlational	0.388	0.747	0.028	0.893	1.000
Guided Backprop	gradient / correlational	0.374	0.747	0.000	0.893	1.000
Linear probing + control tasks	probing	0.362	n/a	n/a	n/a	0.800
A4 · Pairwise correlations	gradient / correlational	0.272	n/a	n/a	0.345	0.107
A3 · Tuning curves	gradient / correlational	0.188	n/a	n/a	0.130	0.875
Attribution / edge patching	gradient / correlational	0.174	n/a	n/a	0.917	1.000
A1 · Connectomics / data-flow graph	causal / intervention	0.000	n/a	n/a	0.981	n/a
A6 · Granger causality	gradient / correlational	0.000	n/a	n/a	0.853	0.150
NMF/PCA dictionaries	dim_reduction	0.000	n/a	n/a	0.915	1.000
Path patching / IOI circuit	causal / intervention	0.000	n/a	n/a	n/a	n/a

Source: site_data.json · generated by gen_site_data.py from the committed §R records + leaderboard.json.

Ground truth

Every method's faithfulness on Jamesbond