Vig et al. 2020; ROME, Meng et al. 2022
Activation patching, also called causal mediation, is a core mechanistic-interpretability method. It runs the system once, records an internal value, and then, on a second run, overwrites, or 'patches', that internal value with the one recorded from a different input. The change in the output measures how much that internal value causally mediates the behaviour. In language-model work this is how researchers localise where a fact or a computation lives inside the network. The idea is genuinely causal: it does not correlate, it substitutes a value and observes the effect. We apply it to the VCS by patching a recorded state value, a RAM cell or register, from one run into another and measuring the causal effect on the output. On this machine the operation is special. Clamping a cell to a donor value and re-running the real ROM is literally the same operation the intervention oracle performs. So activation patching cannot disagree with the oracle: it recovers the exact causal-effect table by construction. This makes it the clearest demonstration that where a method's operation is itself a real intervention on a fully observable machine, faithfulness is automatic. The catch, taken up elsewhere, is that recovering the exact effect table is still not the same as recovering what the patched variables mean.
Each point is one intervention site: its exact causal effect from the oracle (x) against the method's recovered/approximate effect (y). Points on the dashed diagonal mean the method recovered the true effect. max_abs_recovered_minus_exact = 0.000 — this example only (pong, state gameplay(seed=0,prefix=90)+15); the audit aggregate is below.
The score is the maximum absolute difference between the recovered patch effect and the exact patch effect from the oracle, taken across every site and every core game. The grading is always against the intervention oracle. This method is exact by construction, so the difference is zero. Clamping a cell to a donor value and re-running the real ROM is literally the same operation the intervention oracle performs, so the recovered effect table cannot disagree with the oracle, and site precision and recall are both perfect. Faithfulness for this method is therefore at the ceiling, level with the oracle-as-method positive control. There is no sprite-position hard-case here: because the method intervenes on the real machine rather than taking a derivative, it works on position outputs exactly as it works on content outputs. The important qualification, which the score does not by itself capture, is that recovering the exact effect table is not the same as recovering what the patched variables mean or how they combine into the computed function. The audit box reports the measured triad. This method is the clearest demonstration that when a method's operation is itself a real intervention on a fully observable machine, faithfulness is automatic.
The score is measured against the §1 intervention oracle — never against another interpretability method. F (faithful) is always vs the oracle; see the execution stack. How each of F / S / M is actually computed for this method (and whether it matches the paper) is in the box just below; the numbers are in the In the audit box under it.
The exact formula this method uses for each score, read from its runner, and
whether it matches the paper's §3 (F ∧ S ∧ M triad) definition.
From the committed audit
fsm_math_phaseC.json.
| Axis | Formula (as computed) | What it measures | Matches §3? |
|---|---|---|---|
| F faithfulness | \(F = 1 - \max_u|\,\hat{\Delta y}(u)-\Delta y(u)\,| = 1\) | The patch literally re-runs the exact single-site intervention, so recovered effect equals the oracle effect and the causal-effect agreement is 1. | ✓ matches activation_patching.jl main record (metric max_abs_recovered_minus_exact=0 -> F=1.0); jl:693 position record F = _pearson(rec_px, ex_px). Both are the paper's Phase-C patching F = causal-effect-agreement correlation vs Delta_y(u); here exact by construction. Record: extra.triad.F=1.0 (battery mean F=1.0, n=84). how it's measured — call stack:
|
| S sufficiency | \(S = \dfrac{\#\{\,u_{\mathrm{held}}:\ |\hat{y}-y|\le\varepsilon\,\}}{|\mathrm{held\text{-}out}|}\in[0,1]\) | Held-out predictive sufficiency: fit the attribution->Δy map on a calibration half of the causes, predict the disjoint held-out half, count hits within an ε band. | ✓ matches common/triad_sm.jl sufficiency_score, called via triad_extra_dict (activation_patching.jl). Exactly the paper's held-out predictive estimator on the oracle's per-cause bit-exact re-runs. Record: extra.triad.S (activation_patching_pong.json S=1.0; battery mean S=0.996, n=84). how it's measured — call stack:
|
| M minimality | \(M = |U^{\star}| / |\hat{U}| \in (0,1]\quad U^{\star}=\{u:\Delta y(u)>0\},\ \hat{U}=\{u:|\mathrm{attr}(u)|>\tau\}\) | True minimal cause set size over the method-named set size. | ✓ matches common/triad_sm.jl minimality_score. Exactly the paper's M. Record: extra.triad.M=1.0, M_true_minimal_size / M_named_size (pong |U*|=6/|U_hat|=6; battery mean M=0.973, n=84). how it's measured — call stack:
|
✓ matches = the same quantity as §3; ◐ partial = the same kind of estimator but a differing detail; ✗ does not match = a different quantity (see the note). Definitions: F = agreement with the oracle's true causal effects Δy(u); S = held-out predictive score in [−1, 1]; M = |U*|/|Û| (true-minimal-set size / named-set size).
This is the method's entry in the actual cross-method audit — scored on the paper's correctness triad, each axis a mean over all 42 scored games (84 committed §R records), not the single example shown above. Tradition: causal. The example figure (Pong) is one of those records.
F faithfulness (scored vs the oracle for every method) · S sufficiency (held-out predictive score in [−1, 1]; a negative value means the explanation predicts held-out interventions worse than the unperturbed baseline; reported for the predictive methods across all three phases where the calibration/held-out split is defined) · M minimality (true-minimal-set / named-set; where the method names a cause set) — n/a otherwise, per the paper's F ∧ S ∧ M triad.
Source: leaderboard.json · the whole leaderboard is on the
methods page and the
Paper 2 audit.
This method's faithfulness on each of the 42 scored games (all-regime F, and the content vs position split). Click a header to
sort. Every number is read from site_data.json
(methods.activation_patching.per_game).
| Game | F (all) | content-F | position-F | S | M | Note |
|---|---|---|---|---|---|---|
| Air Raid | 1.000 | n/a | 1.000 | 1.000 | 1.000 | recovers the true causes |
| Alien | 1.000 | n/a | 1.000 | 0.833 | 1.000 | recovers the true causes |
| Assault | 1.000 | n/a | 1.000 | 1.000 | 1.000 | recovers the true causes |
| Atlantis | 1.000 | n/a | 1.000 | 1.000 | 0.944 | recovers the true causes |
| Bank Heist | 1.000 | n/a | 1.000 | 1.000 | 1.000 | recovers the true causes |
| Beam Rider | 1.000 | n/a | 1.000 | 1.000 | 1.000 | recovers the true causes |
| Berzerk | 1.000 | n/a | 1.000 | 1.000 | 0.944 | recovers the true causes |
| Bowling | 1.000 | n/a | 1.000 | 1.000 | 0.955 | recovers the true causes |
| Boxing | 1.000 | n/a | 1.000 | 1.000 | 1.000 | recovers the true causes |
| Breakout | 1.000 | n/a | 1.000 | 1.000 | 1.000 | recovers the true causes |
| Carnival | 1.000 | n/a | 1.000 | 1.000 | 0.975 | recovers the true causes |
| Centipede | 1.000 | n/a | 1.000 | 1.000 | 0.958 | recovers the true causes |
| Chopper Command | 1.000 | n/a | 1.000 | 1.000 | 0.964 | recovers the true causes |
| Demon Attack | 1.000 | n/a | 1.000 | 1.000 | 0.929 | recovers the true causes |
| Double Dunk | 1.000 | n/a | 1.000 | 1.000 | 0.971 | recovers the true causes |
| Fishing Derby | 1.000 | n/a | 1.000 | 1.000 | 1.000 | recovers the true causes |
| Freeway | 1.000 | n/a | 1.000 | 1.000 | 1.000 | recovers the true causes |
| Frostbite | 1.000 | n/a | 1.000 | 1.000 | 0.900 | recovers the true causes |
| Gopher | 1.000 | n/a | 1.000 | 1.000 | 1.000 | recovers the true causes |
| Hero | 1.000 | n/a | 1.000 | 1.000 | 0.955 | recovers the true causes |
| Ice Hockey | 1.000 | n/a | 1.000 | 1.000 | 1.000 | recovers the true causes |
| Jamesbond | 1.000 | n/a | 1.000 | 1.000 | 0.909 | recovers the true causes |
| Kangaroo | 1.000 | n/a | 1.000 | 1.000 | 0.909 | recovers the true causes |
| Krull | 1.000 | n/a | 1.000 | 1.000 | 0.923 | recovers the true causes |
| Kung-Fu Master | 1.000 | n/a | 1.000 | 1.000 | 1.000 | recovers the true causes |
| Montezuma's Revenge | 1.000 | n/a | 1.000 | 1.000 | 0.938 | recovers the true causes |
| Ms. Pac-Man | 1.000 | n/a | 1.000 | 1.000 | 1.000 | recovers the true causes |
| Name This Game | 1.000 | n/a | 1.000 | 1.000 | 1.000 | recovers the true causes |
| Pacman | 1.000 | n/a | 1.000 | 1.000 | 1.000 | recovers the true causes |
| Phoenix | 1.000 | n/a | 1.000 | 1.000 | 1.000 | recovers the true causes |
| Pitfall | 1.000 | n/a | 1.000 | 1.000 | 1.000 | recovers the true causes |
| Pong | 1.000 | n/a | 1.000 | 1.000 | 1.000 | recovers the true causes |
| Private Eye | 1.000 | n/a | 1.000 | 1.000 | 1.000 | recovers the true causes |
| Q*bert | 1.000 | n/a | 1.000 | 1.000 | 0.944 | recovers the true causes |
| Riverraid | 1.000 | n/a | 1.000 | 1.000 | 0.940 | recovers the true causes |
| Road Runner | 1.000 | n/a | 1.000 | 1.000 | 0.938 | recovers the true causes |
| Seaquest | 1.000 | n/a | 1.000 | 1.000 | 1.000 | recovers the true causes |
| Space Invaders | 1.000 | n/a | 1.000 | 1.000 | 0.958 | recovers the true causes |
| Tennis | 1.000 | n/a | 1.000 | 1.000 | 0.929 | recovers the true causes |
| Venture | 1.000 | n/a | 1.000 | 1.000 | 0.978 | recovers the true causes |
| Video Pinball | 1.000 | n/a | 1.000 | 1.000 | 1.000 | recovers the true causes |
| Yars' Revenge | 1.000 | n/a | 1.000 | 1.000 | 1.000 | recovers the true causes |
The path from this method's runner (run_game) into the bit-exact VCS substrate, and the
computation it involves. Every step links to the exact source on main. From
callstack_phaseC.json.
env_reset!(env; boot_noop_steps = 60, boot_reset_steps = 4) jutarienv_step!(env::StellaEnvironment, action::Integer) jutaribuild_shared_testbed(game; ... intervene_ram = intervene_ram!, run_intervention = run_intervention, soft_ram_peek = soft_ram_peek, ...) runnersnapshot(env::StellaEnvironment, frame::Integer) jutarirun_patch(checkpoint, tail, kind, site, value) -> intervene_ram!/intervene_tia! runner intervene_ram! @ activation_patching.jl:285, intervene_tia! @ :287rec_snap = run_patch(clean_ckpt, ...); ex_snap = run_patch(fresh_ckpt, ...) inside _run_game_body runner rec_snap @ :460, ex_snap @ :466Scored against the exact intervention oracle Δy(u): run_intervention(checkpoint, actions, target_frame, horizon, cause).
The figure is generated from the committed record by
docs/gen_method_figures.py; the game frame
and each RAM cell's screen footprint are produced by
render_scenes.jl /
cell_footprints.jl.