Every interpretability method in the benchmark, with a link to its implementation, its reference, and the score it is graded on — plus how a run executes, what it writes, and where the records live. This is the Paper-2 counterpart of the conformance code tour.
Execution stack · Oracle & T3 · Phase A · Phase B (attribution) · Phase C (mechanistic) · Leaderboard & benchmark
Because the VCS is fully known and exactly intervenable, every explanation is scored
against the truth. The intervention oracle records the exact causal effect
|Δy(u)| of each candidate cause by clamping it and re-running bit-exact. Each method
runner then produces its own attribution/circuit and is graded against that oracle on the
correctness triad — F faithful (true causes), S sufficient (predicts held-out
interventions), M minimal/right-level. Runners are Julia on the jutari substrate
(jaxtari eager is ≈205× slower), seed = 0, on the 42 scored games,
inside the Paper-1 bit-exact horizon.
ROM (AutoROM, SHA-256 verified — not redistributed)
│ jutari substrate · seed 0 · state f120+30 · games: labeled (42 scored)
▼
ground_truth/oracle_intervene.jl ── exact |Δy(u)| ──▶ the §1 oracle (T1 causal truth)
│
├─ phaseA_kording/A1..A8.jl neuroscience battery
├─ phaseB_attribution/*.jl 12 XAI methods + N/A audit
└─ phaseC_mechanistic/*.jl 10 mechanistic-interp methods
each runner scores its output vs the oracle on the triad F ∧ S ∧ M
└─▶ writes a §R record out//_.json (+ .npz)
▼
compare/leaderboard.py ── pure read of every record ──▶ faithfulness × plausibility
compare/benchmark/run.py ── ROM-free scoring of a new method against the committed oracle Every runner writes a self-describing §R record per game/regime —
{paper, phase, method, game, state, target_output, metric_name, value, ci, n,
seed, where, commit, oracle_ref, timestamp} plus an extra{} block
carrying the exact oracle_abs_delta_per_cause map and the triad
{F,S,M}. That is what makes the leaderboard a pure read and the
benchmark ROM-free.
| Module | Role |
|---|---|
common/results.py | writes/reads the §R record schema |
common/jutari_oracle.jl | the intervention/gradient oracle on the jutari substrate |
common/replay.py | deterministic replay-to-state (the f<start>+<window> encoding) |
common/seeds.py | the single seed=0 source |
common/game_set.json | the labeled game set (42 scored games; 6-game core is a subset) |
repro/make_hash_tables.py | SHA-256 ROM + action-stream hashes (verify with --verify) |
The instrument everything is scored against, and the game-concept labels.
| Step | Implementation | What it computes |
|---|---|---|
| Intervention oracle — the primary ground truth | oracle_intervene.jl | occlude/clamp/resample a cause u, re-run bit-exact, record exact |Δy(u)| |
| Gradient oracle (content path) | oracle_grad.jl | ∂y/∂u + Integrated Gradients through the differentiable substrate (content outputs only) |
| Cross-check | oracle_xcheck.jl | correlation of intervention vs gradient; disagreement is reported, not hidden |
| Import candidate labels | import_labels.py | OCAtari / AtariARI RAM→concept candidates |
| Verify by intervention | verify_labels.jl | set the byte, re-render, confirm the object moves → upgrades to verified-causal |
| Discover new labels | discover_labels.jl | RAM↔framebuffer correlation + intervention sweeps |
Oracle records: ground_truth/out — 3. T3 records: t3/out — 17.
Classical neuroscience methods, scored against the true register-transfer account. The calibration baseline: rich structure, low faithfulness.
| Method (reference) | Implementation | Score description | Audit triad (F / S / M) | Page |
|---|---|---|---|---|
| A1 connectomics / data-flow graph | A1_connectomics.jl | precision/recall + graph-edit-distance vs the true read/write graph | F 0.207 ±0.084 / S 0.971 / M 0.967 · 35 games | explain & figure → |
| A2 single-unit lesions | A2_lesions.jl | rank-correlation of importance with the unit's true role; spurious-specific count | F 0.965 ±0.021 / S 0.589 / M 1.000 · 42 games | explain & figure → |
| A3 tuning curves | A3_tuning.jl | spurious-tuning rate (strongly-tuned units whose tuning ≠ true role) | F 0.213 ±0.080 / S 0.200 / M 0.928 · 42 games | explain & figure → |
| A4 spike-word / pairwise correlations | A4_correlations.jl | weak-pairwise / strong-global structure reproduced vs true coupling | F 0.225 ±0.060 / S 0.133 / M 0.363 · 34 games | explain & figure → |
| A5 local field potentials | A5_lfp.jl | %-variance that is the known clocks (frame/scanline) → epiphenomenal | F 0.376 ±0.055 / S n/a / M n/a · 40 games | explain & figure → |
| A6 Granger causality | A6_granger.jl | false-edge / missed-edge rate vs the true data-flow | F 0.136 ±0.047 / S 0.884 / M 0.647 · 35 games | explain & figure → |
| A7 dim-reduction (NMF/PCA) | A7_dimred.jl | matched-component fraction vs known signals (clock, R/W, vsync) | F 0.519 ±0.061 / S n/a / M n/a · 42 games | explain & figure → |
| A8 whole-state recording | A8_wholestate.jl | descriptive baseline | F 1.000 ±0.000 / S 1.000 / M 0.075 · 42 games | explain & figure → |
F faithfulness (scored vs the oracle for every method) · S sufficiency (held-out predictive score in [−1, 1]; a negative value means the explanation predicts held-out interventions worse than the unperturbed baseline; reported for the predictive methods across all three phases where the calibration/held-out split is defined) · M minimality (true-minimal-set / named-set; where the method names a cause set) — n/a otherwise, per the paper's F ∧ S ∧ M triad.
Records: phaseA_kording/out — 54 committed §R JSON+npz.
One runner per method; each saliency/attribution map scored against the oracle. References are the methods' original papers.
| Method (reference) | Implementation | Score description | Audit triad (F / S / M) | Page |
|---|---|---|---|---|
| Vanilla gradient Simonyan et al. 2014 | saliency.jl | corr + deletion/insertion AUC + precision@k vs true causal top-k | F 0.380 ±0.086 / S 0.593 / M 0.980 · 42 games | explain & figure → |
| Grad×Input / DeepLIFT Shrikumar et al. 2017 | gradxinput.jl | as above + completeness where defined | F 0.486 ±0.090 / S 0.625 / M 0.993 · 42 games | explain & figure → |
| Guided Backprop Springenberg et al. 2015 | guided_backprop.jl | as above + the Adebayo et al. 2018 sanity check | F 0.322 ±0.087 / S 0.573 / M 0.964 · 42 games | explain & figure → |
| SmoothGrad Smilkov et al. 2017 | smoothgrad.jl | noise-averaged saliency; corr + del/ins | F 0.380 ±0.086 / S 0.597 / M 0.981 · 42 games | explain & figure → |
| Integrated Gradients Sundararajan et al. 2017 | ig_baseline_sweep.jl | corr + del/ins + completeness; baseline sweep | F 0.379 ±0.086 / S 0.591 / M 0.984 · 42 games | explain & figure → |
| Expected Gradients Erion et al. 2021 (NMI) | expected_gradients.jl | baseline-averaged IG; as above | F 0.175 ±0.070 / S 0.576 / M 1.000 · 42 games | explain & figure → |
| Occlusion Zeiler & Fergus 2014 | occlusion.jl | del/ins AUC ≈ coarse intervention oracle | F 0.687 ±0.065 / S 0.667 / M 0.973 · 42 games | explain & figure → |
| Extremal / meaningful perturbation Fong & Vedaldi 2017; Fong et al. 2019 | perturbation.jl | learned minimal mask; IoU vs true causal set | F 0.461 ±0.059 / S 0.630 / M 0.969 · 42 games | explain & figure → |
| RISE Petsiuk et al. 2018 | rise.jl | randomized-mask saliency (N=500 masks); corr + del/ins | F 0.552 ±0.065 / S 0.583 / M 0.347 · 42 games | explain & figure → |
| LIME Ribeiro et al. 2016 | lime.jl | local linear weights; corr vs true; stability | F 0.644 ±0.065 / S 0.655 / M 0.557 · 42 games | explain & figure → |
| KernelSHAP / Shapley Lundberg & Lee 2017; Štrumbelj & Kononenko 2014 | kernelshap.jl | Shapley values; corr vs true; convergence vs compute | F 0.652 ±0.065 / S 0.657 / M 0.570 · 42 games | explain & figure → |
| On-distribution counterfactual cf. Olson 2021; Atrey 2020 | counterfactual.jl | minimal valid edit: validity + minimality vs true minimal set | F 0.433 ±0.081 / S 0.639 / M 0.989 · 42 games | explain & figure → |
| N/A audit: Grad-CAM/++, attention rollout, VIPER Selvaraju 2017; Abnar & Zuidema 2020; Bastani 2018 | na_audit.jl | recorded as does not apply (needs NN layers / a policy) | excluded | explain & figure → |
F faithfulness (scored vs the oracle for every method) · S sufficiency (held-out predictive score in [−1, 1]; a negative value means the explanation predicts held-out interventions worse than the unperturbed baseline; reported for the predictive methods across all three phases where the calibration/held-out split is defined) · M minimality (true-minimal-set / named-set; where the method names a cause set) — n/a otherwise, per the paper's F ∧ S ∧ M triad.
Records: phaseB_attribution/out — 166 committed §R JSON+npz.
The state trajectory is the “activations”, the program's data-flow is the “circuit” — both known exactly, so recovered structure is scored against the truth.
| Method (reference) | Implementation | Score description | Audit triad (F / S / M) | Page |
|---|---|---|---|---|
| Activation patching / causal mediation Vig et al. 2020; ROME, Meng et al. 2022 | activation_patching.jl | recovered effect vs the exact patch; site P/R vs true data-flow | F 1.000 ±0.000 / S 0.996 / M 0.973 · 42 games | explain & figure → |
| Interchange interventions / DAS Geiger et al. 2021, 2023 | das.jl | interchange accuracy; alignment vs the true variable | F 1.000 ±0.000 / S 1.000 / M 0.889 · 42 games | explain & figure → |
| Attribution / edge patching Nanda 2023; Syed et al. 2023 | attribution_patching.jl | approx error vs true patching; edge P/R | F 0.456 ±0.113 / S 0.769 / M 0.995 · 42 games | explain & figure → |
| Path patching / IOI circuit Wang et al. 2022; Goldowsky-Dill et al. 2023 | path_patching.jl | circuit precision/recall vs the true routine | F 0.580 ±0.133 / S n/a / M 1.000 · 42 games | explain & figure → |
| ACDC — automatic circuit discovery Conmy et al. 2023 | acdc.jl | edge P/R + scrubbing-preserved performance vs true data-flow | F 0.625 ±0.110 / S 0.299 / M 1.000 · 35 games | explain & figure → |
| Sparse autoencoders Cunningham et al. 2023; Bricken et al. 2023; Templeton et al. 2024 | sae.jl | feature↔known-variable match (F1/MI) + causal use + monosemanticity | F 0.153 ±0.061 / S 0.183 / M 0.934 · 40 games | explain & figure → |
| NMF/PCA dictionaries | dictionaries.jl | matched-component fraction vs known variables | F 0.102 ±0.057 / S 0.751 / M 0.966 · 42 games | explain & figure → |
| Causal scrubbing Chan et al. 2022 | causal_scrubbing.jl | scrubbing-preserved performance vs the true routine | F 0.979 ±0.036 / S 0.979 / M 0.814 · 42 games | explain & figure → |
| Linear probing + control tasks Alain & Bengio 2017; Hewitt & Liang 2019 | linear_probing.jl | accuracy and selectivity (probe − control) → present-vs-used gap | F 0.240 ±0.098 / S n/a / M 0.932 · 42 games | explain & figure → |
| Logit / tuned lens nostalgebraist 2020; Belrose et al. 2023 | logit_lens.jl | readout fidelity vs the true intermediate value | F 0.820 ±0.053 / S n/a / M 0.224 · 42 games | explain & figure → |
F faithfulness (scored vs the oracle for every method) · S sufficiency (held-out predictive score in [−1, 1]; a negative value means the explanation predicts held-out interventions worse than the unperturbed baseline; reported for the predictive methods across all three phases where the calibration/held-out split is defined) · M minimality (true-minimal-set / named-set; where the method names a cause set) — n/a otherwise, per the paper's F ∧ S ∧ M triad.
Records: phaseC_mechanistic/out — 72 committed §R JSON+npz.
| Step | Implementation | Output |
|---|---|---|
| Cross-tradition leaderboard (pure read of all records) | compare/leaderboard.py |
compare/out/leaderboard.json |
| Headline faithful-vs-plausible demo | compare/faithful_demo.py | compare/out/faithful_demo.json |
| Packaged benchmark — score a new method, no ROM needed | compare/benchmark/run.py |
compare/benchmark/out (14) |
The leaderboard re-orients each record's faithfulness onto the headline plot (faithfulness X vs a transparent plausibility proxy Y) and runs an embedded self-check; it never re-runs an experiment. The benchmark scores one method end-to-end against the committed oracle records — a third party needs no ROM.
# the oracle, then any method runner (Julia on the jutari substrate, seed 0)
julia --project=jutari tools/xai_study/ground_truth/oracle_intervene.jl --game pong
julia --project=jutari tools/xai_study/phaseB_attribution/ig_baseline_sweep.jl --games core
julia --project=jutari tools/xai_study/phaseC_mechanistic/activation_patching.jl --games core
# aggregate (pure reads — no ROM) and score a new method
python3 tools/xai_study/compare/leaderboard.py
python3 tools/xai_study/compare/benchmark/run.py --method magnitude_proxyFull per-phase command list: REPRODUCIBILITY.md §3.