Cunningham et al. 2023; Bricken et al. 2023; Templeton et al. 2024
A sparse autoencoder tries to break a tangled state into clean, single-meaning features. It is a small neural network trained to reconstruct the recorded state through a wide but sparsely active hidden layer, so that only a few hidden units fire for any given input. The hope is that each hidden unit becomes 'monosemantic', standing for one human-meaningful concept, which would untangle the superposition thought to hide features inside neural activations. Sparse autoencoders are a central, fast-growing tool in mechanistic interpretability. We train one on the recorded RAM trajectory of the VCS and match its learned features to known, verified game-variable cells. On this small state the reconstruction is nearly perfect and every game-variable cell the SAE is asked to match gets a matching feature. By the metric the field usually uses to claim a feature 'is' a concept, the SAE looks like a complete success. The VCS lets us apply a stronger test. We ablate the matched feature, re-run the real ROM, and ask whether the computation it is supposed to name actually changes. Often it does not: the feature is named but causally unused: clamping it and re-running changes nothing. The sparse autoencoder is therefore our sharpest example of a method that scores well on the standard matching metric while failing the causal test that only a known machine can provide.
Left: the frame. Right: the recovered structure/effect against the ground truth (matched components or preserved behaviour), with the RAM cells labelled. feature_variable_matched_fraction = 1.000 — this example only (pong, state gameplay(seed=0,prefix=90)+15); the audit aggregate is below.
The score is the fraction of sparse-autoencoder features that match a known, verified game-variable cell, together with a causal-use check by ablation. We first measure the matched fraction, how many of the concept cells the SAE was asked to represent get a matching feature, and on this small state that fraction is high. That is the metric the field usually uses to claim a feature is a concept. The VCS then applies a stronger test the audit treats as the real faithfulness: we ablate the matched feature, re-run the real ROM, and ask whether the computation it is supposed to name actually changes. Often it does not, so the causal faithfulness is low even though the matched fraction is high, and the sufficiency can even go negative. This split is the point: a feature can be named yet causally unused. There is no sprite-position hard-case here, because the method is scored on features and their ablation, not on a rendered position. Its plausibility proxy is high, which places it in the danger zone of high plausibility with low faithfulness. The grading is always against the intervention oracle, never against the matching metric alone. The audit box reports the measured triad, so you can read the low causal faithfulness directly.
The score is measured against the §1 intervention oracle — never against another interpretability method. F (faithful) is always vs the oracle; see the execution stack. How each of F / S / M is actually computed for this method (and whether it matches the paper) is in the box just below; the numbers are in the In the audit box under it.
The exact formula this method uses for each score, read from its runner, and
whether it matches the paper's §3 (F ∧ S ∧ M triad) definition.
From the committed audit
fsm_math_phaseC.json.
| Axis | Formula (as computed) | What it measures | Matches §3? |
|---|---|---|---|
| F faithfulness | \(F = \rho_{\mathrm{Spearman}}\!\left(\mathrm{attr}_{\mathrm{recovery}},\ \Delta y(u)\right)\) | Spearman correlation between per-cell SAE recovery and the oracle causal importance: paper-shaped causal-effect agreement. | ✓ matches sae.jl (extra.triad.F = _spearman(recovery, oracle.importance)) IS the oracle-causal-agreement correlation, exactly §3's Phase-C causal-effect-agreement F, and leaderboard.py reads extra.triad.F. Record: sae_pong.json extra.triad.F=0.041. (The top-level 'value' field is still the feature_variable_matched_fraction naming stat, but the committed triad F is the correct causal Spearman.) how it's measured — call stack:
|
| S sufficiency | \(S = \rho_{\mathrm{Pearson}}\!\left(\mathrm{attr}_{\mathrm{recovery}},\ \Delta y_{\mathrm{held\text{-}out}}(u)\right) \in [-1,1]\) | Held-out predictive correlation between SAE recovery and the causal effect on an unseen do(base+37) bit-exact re-run; can be negative. | ✓ matches sae.jl (S = _pearson(recovery, oracle.held_out)). A held-out predictive correlation in [-1,1] on a disjoint intervention (paper-conform; negative allowed). Record: extra.triad.S (sae_pong.json S=−0.458; battery mean S=0.041, n=43). how it's measured — call stack:
|
| M minimality | \(M = |U^{\star}| / |\hat{U}| \in (0,1]\quad U^{\star}=\{u:\Delta y(u)>0\},\ \hat{U}=\{u:|\mathrm{attr}(u)|>\tau\}\) | Oracle-mover count over the number of cells the SAE names above threshold. | ✓ matches sae.jl now emits M = |U*|/|U_hat| in extra.triad.M (M_note '|U*|=4 (oracle movers) / |U_hat|=4 (method above-threshold)'), the paper's set-size ratio. Record: sae_pong.json extra.triad.M=1.0. Replaced the former monosemanticity/target-share proxy. how it's measured — call stack:
|
✓ matches = the same quantity as §3; ◐ partial = the same kind of estimator but a differing detail; ✗ does not match = a different quantity (see the note). Definitions: F = agreement with the oracle's true causal effects Δy(u); S = held-out predictive score in [−1, 1]; M = |U*|/|Û| (true-minimal-set size / named-set size).
This is the method's entry in the actual cross-method audit — scored on the paper's correctness triad, each axis a mean over all 40 scored games (40 committed §R records), not the single example shown above. Tradition: dim_reduction. The example figure (Pong) is one of those records.
F faithfulness (scored vs the oracle for every method) · S sufficiency (held-out predictive score in [−1, 1]; a negative value means the explanation predicts held-out interventions worse than the unperturbed baseline; reported for the predictive methods across all three phases where the calibration/held-out split is defined) · M minimality (true-minimal-set / named-set; where the method names a cause set) — n/a otherwise, per the paper's F ∧ S ∧ M triad.
Source: leaderboard.json · the whole leaderboard is on the
methods page and the
Paper 2 audit.
This method's faithfulness on each of the 40 scored games (all-regime F; this method has no content-vs-position split (see the note below)). Click a header to
sort. Every number is read from site_data.json
(methods.sae.per_game).
content-F and position-F are n/a for this method. The content-vs-position split applies only to pixel-attribution methods — those that produce an attribution map over the framebuffer, whose score can be read separately on the smooth content outputs and the discrete position outputs (where the naive gradient is provably zero). This method does not attribute to a pixel output; it is graded by its own fidelity metric (feature_variable_matched_fraction), reported as the F column, so there is no content or position regime to separate.
S is negative on 22 of these games. Sufficiency asks whether the explanation can predict the outcome of held-out interventions it was not fit on; for this method S is the held-out predictive correlation between its claim and the true effect, so it lies in [−1, 1]. A value near 0 means the explanation predicts no better than the unperturbed output; a negative value means it predicts worse — its held-out claim is anti-correlated with the truth (named yet causally unused). The aggregate S is the mean over the scored games.
| Game | F (all) | content-F | position-F | S | M | Note |
|---|---|---|---|---|---|---|
| Name This Game | 0.783 | n/a | n/a | 0.749 | 1.000 | |
| Bowling | 0.584 | n/a | n/a | 0.476 | 1.000 | |
| Jamesbond | 0.580 | n/a | n/a | -0.097 | 0.875 | |
| Yars' Revenge | 0.570 | n/a | n/a | 0.449 | 0.700 | |
| Alien | 0.359 | n/a | n/a | 0.463 | 1.000 | |
| Krull | 0.308 | n/a | n/a | 0.486 | 1.000 | |
| Freeway | 0.286 | n/a | n/a | 0.481 | 1.000 | |
| Frostbite | 0.275 | n/a | n/a | 0.059 | 1.000 | |
| Hero | 0.273 | n/a | n/a | -0.114 | 1.000 | |
| Road Runner | 0.258 | n/a | n/a | 0.540 | 0.462 | |
| Q*bert | 0.252 | n/a | n/a | -0.117 | 0.467 | |
| Double Dunk | 0.221 | n/a | n/a | 0.587 | 1.000 | |
| Private Eye | 0.208 | n/a | n/a | 0.193 | 1.000 | |
| Riverraid | 0.201 | n/a | n/a | 0.286 | 1.000 | |
| Ice Hockey | 0.176 | n/a | n/a | 0.260 | 1.000 | |
| Ms. Pac-Man | 0.169 | n/a | n/a | -0.258 | 1.000 | |
| Tennis | 0.156 | n/a | n/a | -0.233 | 1.000 | |
| Air Raid | 0.137 | n/a | n/a | 0.090 | 1.000 | |
| Assault | 0.114 | n/a | n/a | 0.374 | 0.900 | |
| Berzerk | 0.054 | n/a | n/a | -0.221 | 0.875 | |
| Atlantis | 0.053 | n/a | n/a | -0.508 | 1.000 | |
| Demon Attack | 0.050 | n/a | n/a | 0.448 | 0.750 | no true-cause signal |
| Pong | 0.041 | n/a | n/a | -0.458 | 1.000 | no true-cause signal |
| Bank Heist | 0.000 | n/a | n/a | -0.506 | 1.000 | no true-cause signal |
| Beam Rider | 0.000 | n/a | n/a | -0.262 | 1.000 | no true-cause signal |
| Boxing | 0.000 | n/a | n/a | 0.215 | 1.000 | no true-cause signal |
| Breakout | 0.000 | n/a | n/a | -0.447 | 1.000 | no true-cause signal |
| Carnival | 0.000 | n/a | n/a | -0.112 | 1.000 | no true-cause signal |
| Centipede | 0.000 | n/a | n/a | -0.446 | 1.000 | no true-cause signal |
| Chopper Command | 0.000 | n/a | n/a | -0.086 | 0.769 | no true-cause signal |
| Fishing Derby | 0.000 | n/a | n/a | -0.351 | 0.714 | no true-cause signal |
| Kangaroo | 0.000 | n/a | n/a | -0.073 | 1.000 | no true-cause signal |
| Montezuma's Revenge | 0.000 | n/a | n/a | -0.224 | 1.000 | no true-cause signal |
| Pacman | 0.000 | n/a | n/a | -0.304 | 1.000 | no true-cause signal |
| Phoenix | 0.000 | n/a | n/a | -0.270 | 1.000 | no true-cause signal |
| Pitfall | 0.000 | n/a | n/a | -0.244 | 0.846 | no true-cause signal |
| Seaquest | 0.000 | n/a | n/a | 0.706 | 1.000 | no true-cause signal |
| Space Invaders | 0.000 | n/a | n/a | -0.128 | 1.000 | no true-cause signal |
| Venture | 0.000 | n/a | n/a | -0.035 | 1.000 | no true-cause signal |
| Video Pinball | 0.000 | n/a | n/a | 0.450 | 1.000 | no true-cause signal |
The path from this method's runner (compute_game) into the bit-exact VCS substrate, and the
computation it involves. Every step links to the exact source on main. From
callstack_phaseC.json.
env_reset!(env; boot_noop_steps = 60, boot_reset_steps = 4) jutarienv_step!(env::StellaEnvironment, action::Integer) jutaribuild_shared_testbed(game; ... intervene_ram = J.intervene_ram!, run_intervention = O.run_intervention, soft_ram_peek = soft_ram_peek, ...) runnersnapshot(env::StellaEnvironment, frame::Integer) jutarisae_record_ram(game, frames, ...) -> env_step! runnerfit!(s::SAE, X; epochs, lr) (ctor SAE @ :375, loss @ :395) runnercausal_use(...) -> run_full_write(raw_ablate) -> intervene_ram! over all cells; env_step! runner run_full_write writes the decoded RAM via intervene_ram! (sae.jl:627), denorm_ram @ :598; scored only on ORACLE-CAUSAL cells via oracle_importance @ :536Scored against the exact intervention oracle Δy(u): run_intervention(checkpoint, actions, target_frame, horizon, cause).
The figure is generated from the committed record by
docs/gen_method_figures.py; the game frame
and each RAM cell's screen footprint are produced by
render_scenes.jl /
cell_footprints.jl.