nostalgebraist 2020; Belrose et al. 2023
The logit lens reads a system's intermediate state through its own output decoder. In a transformer it takes the hidden state at some middle layer and applies the final output projection to it, as if that intermediate state were the last layer, to see what the model would 'say' at that point. This reveals how the prediction forms across depth, and the tuned-lens variant learns a small correction so the readout is more faithful at each stage. It is a standard, cheap tool for tracing how information develops inside a network. We build the analogue on the VCS by reading an intermediate value through the machine's own decoding, and comparing the decoded value to the true intermediate value at that site. On this transparent machine the readout is exact: the state is linearly readable at the right site, so the lens reproduces the true intermediate value on every game. This is expected, because the VCS keeps its variables in plain, addressable memory rather than in a distorted distributed code. The logit lens is therefore a clean positive result on the machine, and it sits with the exact causal methods. As with those, reading the right value at the right site confirms that the value is represented and readable, but it does not by itself explain how that value is computed or what role it plays.
Left: the frame. Right: the recovered structure/effect against the ground truth (matched components or preserved behaviour), with the RAM cells labelled. logit_lens_faithful_weight_fraction_vs_causal_mask = 0.438 — this example only (pong, state gameplay(seed=0,prefix=90)+15); the audit aggregate is below.
The score is the readout fidelity of the lens-decoded intermediate against the true intermediate value, reported as an R-squared. We read an intermediate value through the machine's own decoding and compare it to the true value at that site. The grading is always against the intervention oracle. On this transparent machine the readout is exact: the state is linearly readable at the right site, so the lens reproduces the true intermediate value on every core game and the fidelity is at the ceiling. Faithfulness for this method is that readout fidelity. There is no sprite-position hard-case here, because the method reads an internal value rather than a rendered position, and it does not rely on a gradient. The result is expected, because the VCS keeps its variables in plain, addressable memory rather than in a distorted distributed code. The important qualification, which the score does not by itself capture, is that reading the right value at the right site confirms the value is represented and readable, but does not explain how the value is computed or what role it plays. The grading is always against the oracle. The audit box reports the measured triad, with fidelity at the ceiling.
The score is measured against the §1 intervention oracle — never against another interpretability method. F (faithful) is always vs the oracle; see the execution stack. How each of F / S / M is actually computed for this method (and whether it matches the paper) is in the box just below; the numbers are in the In the audit box under it.
The exact formula this method uses for each score, read from its runner, and
whether it matches the paper's §3 (F ∧ S ∧ M triad) definition.
From the committed audit
fsm_math_phaseC.json.
| Axis | Formula (as computed) | What it measures | Matches §3? |
|---|---|---|---|
| F faithfulness | \(F = \dfrac{\sum_t |A_t|\cdot \mathbb{1}[\text{site causal}]}{\sum_t |A_t|}\) | Mean fraction of the tuned-lens readout weight that lands on truly-causal sites (the oracle causal mask): low => the lens reads a spurious correlate (present != used). | ✓ matches logit_lens.jl now emits top-level metric_name=logit_lens_faithful_weight_fraction_vs_causal_mask with extra.triad.F = mean faithful-weight fraction vs the oracle causal mask; it replaced the readout-fidelity-to-the-true-intermediate (=1.0 by construction). This is §3's causal-agreement F for a readout-direction method. Record: logit_lens_pong.json extra.triad.F=0.438. how it's measured — call stack:
|
| S sufficiency | S = null (undefined) | Not scored: the lens reports readout fidelity to the true intermediate, not a held-out do(u) output prediction. | — n/a logit_lens.jl (S = nothing). n/a is defensible per paper. Record: extra.triad.S = null. |
| M minimality | \(M = |U^{\star}| / |\hat{U}| \in (0,1]\quad U^{\star}=\{u:\Delta y(u)>0\},\ \hat{U}=\{u:|\mathrm{attr}(u)|>\tau\}\) | Oracle-causal sites over the number of readout sites the lens names above threshold. | ✓ matches logit_lens.jl now emits M = |U*|/|U_hat| in extra.triad.M (M_note '|U*|=4 (oracle movers) / |U_hat|=74 (method above-threshold)'), the paper's set-size ratio. Record: logit_lens_pong.json extra.triad.M=0.054. Replaced the former faithful-weight-fraction proxy (now used as F). how it's measured — call stack:
|
✓ matches = the same quantity as §3; ◐ partial = the same kind of estimator but a differing detail; ✗ does not match = a different quantity (see the note). Definitions: F = agreement with the oracle's true causal effects Δy(u); S = held-out predictive score in [−1, 1]; M = |U*|/|Û| (true-minimal-set size / named-set size).
This is the method's entry in the actual cross-method audit — scored on the paper's correctness triad, each axis a mean over all 42 scored games (42 committed §R records), not the single example shown above. Tradition: causal. The example figure (Pong) is one of those records.
F faithfulness (scored vs the oracle for every method) · S sufficiency (held-out predictive score in [−1, 1]; a negative value means the explanation predicts held-out interventions worse than the unperturbed baseline; reported for the predictive methods across all three phases where the calibration/held-out split is defined) · M minimality (true-minimal-set / named-set; where the method names a cause set) — n/a otherwise, per the paper's F ∧ S ∧ M triad.
Source: leaderboard.json · the whole leaderboard is on the
methods page and the
Paper 2 audit.
This method's faithfulness on each of the 42 scored games (all-regime F; this method has no content-vs-position split (see the note below)). Click a header to
sort. Every number is read from site_data.json
(methods.logit_lens.per_game).
content-F and position-F are n/a for this method. The content-vs-position split applies only to pixel-attribution methods — those that produce an attribution map over the framebuffer, whose score can be read separately on the smooth content outputs and the discrete position outputs (where the naive gradient is provably zero). This method does not attribute to a pixel output; it is graded by its own fidelity metric (logit_lens_faithful_weight_fraction_vs_causal_mask), reported as the F column, so there is no content or position regime to separate.
| Game | F (all) | content-F | position-F | S | M | Note |
|---|---|---|---|---|---|---|
| Beam Rider | 1.000 | n/a | n/a | n/a | 1.000 | recovers the true causes |
| Gopher | 1.000 | n/a | n/a | n/a | 1.000 | recovers the true causes |
| Kung-Fu Master | 1.000 | n/a | n/a | n/a | 1.000 | recovers the true causes |
| Pacman | 1.000 | n/a | n/a | n/a | 1.000 | recovers the true causes |
| Bowling | 0.995 | n/a | n/a | n/a | 0.298 | recovers the true causes |
| Bank Heist | 0.991 | n/a | n/a | n/a | 0.069 | recovers the true causes |
| Phoenix | 0.984 | n/a | n/a | n/a | 0.116 | recovers the true causes |
| Air Raid | 0.976 | n/a | n/a | n/a | 0.140 | recovers the true causes |
| Carnival | 0.975 | n/a | n/a | n/a | 0.168 | recovers the true causes |
| Q*bert | 0.972 | n/a | n/a | n/a | 0.271 | recovers the true causes |
| Freeway | 0.969 | n/a | n/a | n/a | 0.122 | recovers the true causes |
| Kangaroo | 0.950 | n/a | n/a | n/a | 0.143 | recovers the true causes |
| Name This Game | 0.947 | n/a | n/a | n/a | 0.085 | recovers the true causes |
| Centipede | 0.928 | n/a | n/a | n/a | 0.074 | recovers the true causes |
| Road Runner | 0.908 | n/a | n/a | n/a | 0.165 | recovers the true causes |
| Ice Hockey | 0.891 | n/a | n/a | n/a | 0.096 | recovers the true causes |
| Yars' Revenge | 0.886 | n/a | n/a | n/a | 0.156 | recovers the true causes |
| Private Eye | 0.878 | n/a | n/a | n/a | 0.279 | recovers the true causes |
| Krull | 0.875 | n/a | n/a | n/a | 0.247 | recovers the true causes |
| Venture | 0.862 | n/a | n/a | n/a | 0.182 | recovers the true causes |
| Tennis | 0.860 | n/a | n/a | n/a | 0.169 | recovers the true causes |
| Double Dunk | 0.853 | n/a | n/a | n/a | 0.117 | recovers the true causes |
| Space Invaders | 0.847 | n/a | n/a | n/a | 0.121 | |
| Montezuma's Revenge | 0.842 | n/a | n/a | n/a | 0.119 | |
| Riverraid | 0.842 | n/a | n/a | n/a | 0.183 | |
| Alien | 0.820 | n/a | n/a | n/a | 0.061 | |
| Breakout | 0.815 | n/a | n/a | n/a | 0.067 | |
| Seaquest | 0.812 | n/a | n/a | n/a | 0.310 | |
| Hero | 0.805 | n/a | n/a | n/a | 0.140 | |
| Video Pinball | 0.803 | n/a | n/a | n/a | 0.080 | |
| Demon Attack | 0.791 | n/a | n/a | n/a | 0.151 | |
| Jamesbond | 0.764 | n/a | n/a | n/a | 0.105 | |
| Atlantis | 0.748 | n/a | n/a | n/a | 0.096 | |
| Ms. Pac-Man | 0.734 | n/a | n/a | n/a | 0.168 | |
| Berzerk | 0.662 | n/a | n/a | n/a | 0.209 | |
| Fishing Derby | 0.624 | n/a | n/a | n/a | 0.050 | |
| Frostbite | 0.568 | n/a | n/a | n/a | 0.137 | |
| Chopper Command | 0.553 | n/a | n/a | n/a | 0.111 | |
| Assault | 0.530 | n/a | n/a | n/a | 0.163 | |
| Pitfall | 0.441 | n/a | n/a | n/a | 0.106 | |
| Pong | 0.438 | n/a | n/a | n/a | 0.054 | |
| Boxing | 0.291 | n/a | n/a | n/a | 0.081 |
The path from this method's runner (run_game) into the bit-exact VCS substrate, and the
computation it involves. Every step links to the exact source on main. From
callstack_phaseC.json.
env_reset!(env; boot_noop_steps = 60, boot_reset_steps = 4) jutarienv_step!(env::StellaEnvironment, action::Integer) jutaribuild_shared_testbed(game; ... intervene_ram = intervene_ram!, run_intervention = run_intervention, soft_ram_peek = soft_ram_peek, ...) runnersnapshot(env::StellaEnvironment, frame::Integer) jutaribuild_trajset(base_ckpt, src_ckpt, ...) -> intervene_ram! runner intervene_ram! @ logit_lens.jl:277; record_ram_tape @ :213ridge_fit(X, y; lambda) ; r2(y, yhat) runner r2 @ :300build_causal_mask(base_ckpt, tail, ...) -> intervene_ram! runner intervene_ram! @ logit_lens.jl:379Scored against the exact intervention oracle Δy(u): run_intervention(checkpoint, actions, target_frame, horizon, cause).
The figure is generated from the committed record by
docs/gen_method_figures.py; the game frame
and each RAM cell's screen footprint are produced by
render_scenes.jl /
cell_footprints.jl.