Alain & Bengio 2017; Hewitt & Liang 2019
Linear probing asks whether a concept can be read out of the state by a simple classifier. A linear probe is trained to predict a concept, say the ball's position, from the internal state; if it succeeds, the concept is said to be 'encoded' there. The well-known danger is that a probe can succeed just because the concept happens to be linearly recoverable, not because the system uses it. Hewitt and Liang's control-task fix addresses this: train a second probe to predict a random label, and report selectivity, the probe's real accuracy minus its accuracy on the random control. High selectivity means the state genuinely encodes the concept rather than the probe memorising it. We apply this on the VCS, training probes for game concepts and subtracting the control-task baseline. The VCS then adds a test no neural network allows: we check, by intervention, whether the program actually uses each decodable cell. It often does not. A cell can be decodable, and pass the control-task check, yet be provably unused when we clamp it and re-run, because the value is present in the state but never causally used within the window. Linear probing is thus our clearest illustration of the present-versus-used gap: decodability shows information is there, but only the intervention oracle shows whether the program relies on it.
Per labelled RAM cell, the probe's selectivity (accuracy minus a control task). Red bars are cells that are decodable but not causally used — present ≠ used. probe_selectivity_vs_causal_use_correlation = -0.090 — this example only (pong, state gameplay(seed=0,prefix=90)+15, NOOP-continued to f600); the audit aggregate is below.
The score is the mean selectivity: the probe's accuracy minus the accuracy of a control-task probe trained on a random label, averaged over the labelled cells. The control task follows Hewitt and Liang and guards against a probe succeeding merely because a concept is linearly recoverable. The grading is always against the intervention oracle. Faithfulness for this method is that selectivity, and it is modest, but the deeper finding comes from crossing the probe against the oracle. Concepts are decodable above the control, so the information is present in the state, yet the oracle flags cells that are decodable but never causally used within the window. That is the present-versus-used gap: a value can be read out without the program relying on it. There is no sprite-position hard-case here; the output is a decodability judgement. The grading is always against the oracle, which is strictly stronger than the probe, because the probe asks whether a value can be read while the oracle asks whether changing it changes the output. The audit box reports the measured triad. Linear probing earns a high plausibility proxy on a low faithfulness, which places it in the danger zone and makes it the clearest illustration of decodable-but-unused information.
The score is measured against the §1 intervention oracle — never against another interpretability method. F (faithful) is always vs the oracle; see the execution stack. How each of F / S / M is actually computed for this method (and whether it matches the paper) is in the box just below; the numbers are in the In the audit box under it.
The exact formula this method uses for each score, read from its runner, and
whether it matches the paper's §3 (F ∧ S ∧ M triad) definition.
From the committed audit
fsm_math_phaseC.json.
| Axis | Formula (as computed) | What it measures | Matches §3? |
|---|---|---|---|
| F faithfulness | \(F = \rho_{\mathrm{pb}}\!\left(\mathrm{attr}_{\mathrm{sel}},\ \mathbb{1}[\text{causally used}]\right)\) | Point-biserial correlation between per-cell probe selectivity and the oracle causally-used indicator: does selectivity track causal use (not mere presence)? | ✓ matches linear_probing.jl now emits top-level metric_name=probe_selectivity_vs_causal_use_correlation with extra.triad.F = point-biserial rho(selectivity, oracle causally-used indicator); it replaced the mean control-task selectivity (a decodability/presence stat). This is §3's correlation between the attribution and the true causal effect. Record: linear_probing_pong.json extra.triad.F=-0.090 (present-but-unused => low/negative). how it's measured — call stack:
|
| S sufficiency | S = null (undefined) | Not scored: a probe reads presence, not causal use; it emits no held-out do(u) output prediction. | — n/a linear_probing.jl (S = nothing). Deliberately n/a per paper (present != used, Hewitt & Liang 2019). Record: extra.triad.S = null. |
| M minimality | \(M = |U^{\star}| / |\hat{U}| \in (0,1]\quad U^{\star}=\{u:\Delta y(u)>0\},\ \hat{U}=\{u:|\mathrm{attr}(u)|>\tau\}\) | Oracle causally-used cells over the number of cells the probe names by selectivity. | ✓ matches linear_probing.jl now emits M = |U*|/|U_hat| in extra.triad.M (M_note '|U*|=3 (oracle movers) / |U_hat|=5 (method above-threshold)'), the paper's set-size ratio. Record: linear_probing_pong.json extra.triad.M=0.6. how it's measured — call stack:
|
✓ matches = the same quantity as §3; ◐ partial = the same kind of estimator but a differing detail; ✗ does not match = a different quantity (see the note). Definitions: F = agreement with the oracle's true causal effects Δy(u); S = held-out predictive score in [−1, 1]; M = |U*|/|Û| (true-minimal-set size / named-set size).
This is the method's entry in the actual cross-method audit — scored on the paper's correctness triad, each axis a mean over all 42 scored games (42 committed §R records), not the single example shown above. Tradition: probing. The example figure (Pong) is one of those records.
F faithfulness (scored vs the oracle for every method) · S sufficiency (held-out predictive score in [−1, 1]; a negative value means the explanation predicts held-out interventions worse than the unperturbed baseline; reported for the predictive methods across all three phases where the calibration/held-out split is defined) · M minimality (true-minimal-set / named-set; where the method names a cause set) — n/a otherwise, per the paper's F ∧ S ∧ M triad.
Source: leaderboard.json · the whole leaderboard is on the
methods page and the
Paper 2 audit.
This method's faithfulness on each of the 27 scored games (all-regime F; this method has no content-vs-position split (see the note below)). Click a header to
sort. Every number is read from site_data.json
(methods.linear_probing.per_game).
content-F and position-F are n/a for this method. The content-vs-position split applies only to pixel-attribution methods — those that produce an attribution map over the framebuffer, whose score can be read separately on the smooth content outputs and the discrete position outputs (where the naive gradient is provably zero). This method does not attribute to a pixel output; it is graded by its own fidelity metric (probe_selectivity_vs_causal_use_correlation), reported as the F column, so there is no content or position regime to separate.
M is n/a on 1 of these games. Minimality is M = |U*|/|Û|, the size of the true minimal cause set over the size of the set the method actually names. On a game where the method names nothing (|Û| = 0 — it discovers no circuit, its attribution map is all-zero, or it decodes no cell), that ratio is undefined, so M is left blank rather than scored. The aggregate M is the mean over the 26 games where the method did name a cause set.
| Game | F (all) | content-F | position-F | S | M | Note |
|---|---|---|---|---|---|---|
| Beam Rider | 1.000 | n/a | n/a | n/a | 0.500 | recovers the true causes |
| Q*bert | 1.000 | n/a | n/a | n/a | n/a | recovers the true causes |
| Carnival | 0.419 | n/a | n/a | n/a | 1.000 | |
| Ms. Pac-Man | 0.410 | n/a | n/a | n/a | 1.000 | |
| Jamesbond | 0.362 | n/a | n/a | n/a | 0.800 | |
| Breakout | 0.354 | n/a | n/a | n/a | 1.000 | |
| Centipede | 0.341 | n/a | n/a | n/a | 1.000 | |
| Montezuma's Revenge | 0.336 | n/a | n/a | n/a | 1.000 | |
| Kangaroo | 0.328 | n/a | n/a | n/a | 1.000 | |
| Assault | 0.288 | n/a | n/a | n/a | 1.000 | |
| Road Runner | 0.288 | n/a | n/a | n/a | 1.000 | |
| Frostbite | 0.275 | n/a | n/a | n/a | 1.000 | |
| Pitfall | 0.184 | n/a | n/a | n/a | 1.000 | |
| Yars' Revenge | 0.152 | n/a | n/a | n/a | 1.000 | |
| Space Invaders | 0.145 | n/a | n/a | n/a | 0.500 | |
| Fishing Derby | 0.143 | n/a | n/a | n/a | 0.667 | |
| Demon Attack | 0.123 | n/a | n/a | n/a | 1.000 | |
| Riverraid | 0.118 | n/a | n/a | n/a | 1.000 | |
| Phoenix | 0.116 | n/a | n/a | n/a | 1.000 | |
| Chopper Command | 0.064 | n/a | n/a | n/a | 0.571 | |
| Berzerk | 0.043 | n/a | n/a | n/a | 1.000 | no true-cause signal |
| Atlantis | 0.000 | n/a | n/a | n/a | 1.000 | no true-cause signal |
| Boxing | 0.000 | n/a | n/a | n/a | 0.625 | no true-cause signal |
| Ice Hockey | 0.000 | n/a | n/a | n/a | 1.000 | no true-cause signal |
| Pong | 0.000 | n/a | n/a | n/a | 0.600 | no true-cause signal |
| Private Eye | 0.000 | n/a | n/a | n/a | 1.000 | no true-cause signal |
| Tennis | 0.000 | n/a | n/a | n/a | 1.000 | no true-cause signal |
The path from this method's runner (run_game) into the bit-exact VCS substrate, and the
computation it involves. Every step links to the exact source on main. From
callstack_phaseC.json.
env_reset!(env; boot_noop_steps = 60, boot_reset_steps = 4) jutarienv_step!(env::StellaEnvironment, action::Integer) jutaribuild_shared_testbed(game; ... intervene_ram = J.intervene_ram!, run_intervention = O.run_intervention, soft_ram_peek = soft_ram_peek, ...) runnersnapshot(env::StellaEnvironment, frame::Integer) jutarirecord_ram_tape(game, actions) -> env_step! runnerprobe_accuracy(Xtr, ytr, Xte, yte, nclass; lambda = RIDGE_LAMBDA) runneroracle_transient_cells(game::AbstractString) runnerScored against the exact intervention oracle Δy(u): run_intervention(checkpoint, actions, target_frame, horizon, cause).
The figure is generated from the committed record by
docs/gen_method_figures.py; the game frame
and each RAM cell's screen footprint are produced by
render_scenes.jl /
cell_footprints.jl.