Geiger et al. 2021, 2023
Distributed Alignment Search, or DAS, looks for a high-level concept inside the low-level state. It learns a rotation of the internal representation so that a chosen direction, or subspace, lines up with a supplied causal variable. It then tests the alignment by an interchange intervention: swap that subspace between two runs and check whether the output changes exactly as the concept would predict. The method is a leading tool for arguing that a network represents an abstract variable in a specific, possibly distributed, part of its activations. We apply it to the VCS by aligning a learned subspace of the state to a known game variable and measuring interchange accuracy against the true variable. On this transparent machine the alignment is exact: swapping the aligned subspace reproduces the concept's effect perfectly, while a mis-aligned control does not. There is an important caveat that the VCS makes explicit. The concept DAS aligns to is one we supplied from an external label, not one the method discovered. So a perfect interchange accuracy confirms an alignment to a given meaning; it does not show the method found the meaning on its own. DAS is thus a strong test of whether a supplied concept is causally real in the state, and a clean example of the difference between verifying a label and discovering one.
Each point is one intervention site: its exact causal effect from the oracle (x) against the method's recovered/approximate effect (y). Points on the dashed diagonal mean the method recovered the true effect. interchange_accuracy_aligned = 1.000 — this example only (pong, state gameplay(seed=0,prefix=90)+15); the audit aggregate is below.
The score is the interchange accuracy of the aligned subspace against the true variable. We align a learned subspace of the state to a supplied concept, then swap that subspace between two runs and check whether the output changes exactly as the concept predicts; the fraction of correct interchanges is the accuracy. The grading is against the intervention oracle, and on this transparent machine the alignment is exact, so interchange accuracy is at the ceiling, while a mis-aligned control scores zero. Faithfulness for this method is that interchange accuracy. There is no sprite-position hard-case here, because DAS operates by intervention rather than by a gradient. The essential caveat the audit makes explicit is that the concept DAS aligns to is one we supplied from an external label, not one the method discovered. So a perfect interchange accuracy is guaranteed in advance: it confirms an alignment to a given meaning, and does not show the method found the meaning on its own. The grading is always against the oracle, never against another method. The audit box reports the measured triad. DAS is thus a clean example of the difference between verifying a supplied concept and discovering one, which only a machine with known ground truth can separate.
The score is measured against the §1 intervention oracle — never against another interpretability method. F (faithful) is always vs the oracle; see the execution stack. How each of F / S / M is actually computed for this method (and whether it matches the paper) is in the box just below; the numbers are in the In the audit box under it.
The exact formula this method uses for each score, read from its runner, and
whether it matches the paper's §3 (F ∧ S ∧ M triad) definition.
From the committed audit
fsm_math_phaseC.json.
| Axis | Formula (as computed) | What it measures | Matches §3? |
|---|---|---|---|
| F faithfulness | \(F = \mathrm{interchange\ accuracy} = 1\) | Interchange accuracy of the SUPPLIED alignment: for each aligned cell, does swapping its value reproduce the predicted high-level outcome; 1.0 by construction on the VCS. | ◐ partial das.jl (F = clamp(iia_aligned)); metric_name interchange_accuracy_aligned. It IS a causal check (each interchange is a real do(cell:=source) bit-exact re-run, so single-cell interchange == the exact patch oracle), but F verifies a SUPPLIED/known-true alignment reproduces the abstraction (Geiger IIA) rather than scoring a method-discovered claim's agreement against {Delta_y(u)} -> '1.0 by construction'. The genuine discovery signal (alignment_accuracy, whether DAS recovers the right cell) is a separate extra field, not F. Record: extra.triad.F=1.0 (battery mean F=1.0, n=42). how it's measured — call stack:
|
| S sufficiency | \(S = \dfrac{\#\{\,u_{\mathrm{held}}:\ |\hat{y}-y|\le\varepsilon\,\}}{|\mathrm{held\text{-}out}|}\in[0,1]\) | Shared held-out predictive sufficiency estimator over the interchange effects. | ✓ matches common/triad_sm.jl sufficiency_score via triad_extra_dict (das.jl, effect vs exact_effect). Paper-conform. Record: extra.triad.S=1.0 (battery mean S=1.0, n=42). how it's measured — call stack:
|
| M minimality | \(M = |U^{\star}| / |\hat{U}| \in (0,1]\quad U^{\star}=\{u:\Delta y(u)>0\},\ \hat{U}=\{u:|\mathrm{attr}(u)|>\tau\}\) | True minimal cause set over method-named set. | ✓ matches common/triad_sm.jl minimality_score. Paper-conform. Record: extra.triad.M (pong |U*|=5/|U_hat|=5, M=1.0; battery mean M=0.890, n=42). how it's measured — call stack:
|
✓ matches = the same quantity as §3; ◐ partial = the same kind of estimator but a differing detail; ✗ does not match = a different quantity (see the note). Definitions: F = agreement with the oracle's true causal effects Δy(u); S = held-out predictive score in [−1, 1]; M = |U*|/|Û| (true-minimal-set size / named-set size).
This is the method's entry in the actual cross-method audit — scored on the paper's correctness triad, each axis a mean over all 42 scored games (42 committed §R records), not the single example shown above. Tradition: causal. The example figure (Pong) is one of those records.
F faithfulness (scored vs the oracle for every method) · S sufficiency (held-out predictive score in [−1, 1]; a negative value means the explanation predicts held-out interventions worse than the unperturbed baseline; reported for the predictive methods across all three phases where the calibration/held-out split is defined) · M minimality (true-minimal-set / named-set; where the method names a cause set) — n/a otherwise, per the paper's F ∧ S ∧ M triad.
Source: leaderboard.json · the whole leaderboard is on the
methods page and the
Paper 2 audit.
This method's faithfulness on each of the 42 scored games (all-regime F; this method has no content-vs-position split (see the note below)). Click a header to
sort. Every number is read from site_data.json
(methods.das.per_game).
content-F and position-F are n/a for this method. The content-vs-position split applies only to pixel-attribution methods — those that produce an attribution map over the framebuffer, whose score can be read separately on the smooth content outputs and the discrete position outputs (where the naive gradient is provably zero). This method does not attribute to a pixel output; it is graded by its own fidelity metric (interchange_accuracy_aligned), reported as the F column, so there is no content or position regime to separate.
| Game | F (all) | content-F | position-F | S | M | Note |
|---|---|---|---|---|---|---|
| Air Raid | 1.000 | n/a | n/a | 1.000 | 1.000 | recovers the true causes |
| Alien | 1.000 | n/a | n/a | 1.000 | 0.857 | recovers the true causes |
| Assault | 1.000 | n/a | n/a | 1.000 | 0.875 | recovers the true causes |
| Atlantis | 1.000 | n/a | n/a | 1.000 | 0.889 | recovers the true causes |
| Bank Heist | 1.000 | n/a | n/a | 1.000 | 1.000 | recovers the true causes |
| Beam Rider | 1.000 | n/a | n/a | 1.000 | 1.000 | recovers the true causes |
| Berzerk | 1.000 | n/a | n/a | 1.000 | 0.902 | recovers the true causes |
| Bowling | 1.000 | n/a | n/a | 1.000 | 0.909 | recovers the true causes |
| Boxing | 1.000 | n/a | n/a | 1.000 | 0.650 | recovers the true causes |
| Breakout | 1.000 | n/a | n/a | 1.000 | 1.000 | recovers the true causes |
| Carnival | 1.000 | n/a | n/a | 1.000 | 0.895 | recovers the true causes |
| Centipede | 1.000 | n/a | n/a | 1.000 | 0.900 | recovers the true causes |
| Chopper Command | 1.000 | n/a | n/a | 1.000 | 0.647 | recovers the true causes |
| Demon Attack | 1.000 | n/a | n/a | 1.000 | 0.857 | recovers the true causes |
| Double Dunk | 1.000 | n/a | n/a | 1.000 | 0.895 | recovers the true causes |
| Fishing Derby | 1.000 | n/a | n/a | 1.000 | 1.000 | recovers the true causes |
| Freeway | 1.000 | n/a | n/a | 1.000 | 1.000 | recovers the true causes |
| Frostbite | 1.000 | n/a | n/a | 1.000 | 0.593 | recovers the true causes |
| Gopher | 1.000 | n/a | n/a | 1.000 | 1.000 | recovers the true causes |
| Hero | 1.000 | n/a | n/a | 1.000 | 0.833 | recovers the true causes |
| Ice Hockey | 1.000 | n/a | n/a | 1.000 | 1.000 | recovers the true causes |
| Jamesbond | 1.000 | n/a | n/a | 1.000 | 0.769 | recovers the true causes |
| Kangaroo | 1.000 | n/a | n/a | 1.000 | 0.769 | recovers the true causes |
| Krull | 1.000 | n/a | n/a | 1.000 | 0.929 | recovers the true causes |
| Kung-Fu Master | 1.000 | n/a | n/a | 1.000 | 1.000 | recovers the true causes |
| Montezuma's Revenge | 1.000 | n/a | n/a | 1.000 | 0.731 | recovers the true causes |
| Ms. Pac-Man | 1.000 | n/a | n/a | 1.000 | 1.000 | recovers the true causes |
| Name This Game | 1.000 | n/a | n/a | 1.000 | 1.000 | recovers the true causes |
| Pacman | 1.000 | n/a | n/a | 1.000 | 1.000 | recovers the true causes |
| Phoenix | 1.000 | n/a | n/a | 1.000 | 1.000 | recovers the true causes |
| Pitfall | 1.000 | n/a | n/a | 1.000 | 0.923 | recovers the true causes |
| Pong | 1.000 | n/a | n/a | 1.000 | 1.000 | recovers the true causes |
| Private Eye | 1.000 | n/a | n/a | 1.000 | 0.917 | recovers the true causes |
| Q*bert | 1.000 | n/a | n/a | 1.000 | 0.950 | recovers the true causes |
| Riverraid | 1.000 | n/a | n/a | 1.000 | 0.774 | recovers the true causes |
| Road Runner | 1.000 | n/a | n/a | 1.000 | 0.750 | recovers the true causes |
| Seaquest | 1.000 | n/a | n/a | 1.000 | 0.970 | recovers the true causes |
| Space Invaders | 1.000 | n/a | n/a | 1.000 | 0.867 | recovers the true causes |
| Tennis | 1.000 | n/a | n/a | 1.000 | 0.708 | recovers the true causes |
| Venture | 1.000 | n/a | n/a | 1.000 | 0.727 | recovers the true causes |
| Video Pinball | 1.000 | n/a | n/a | 1.000 | 1.000 | recovers the true causes |
| Yars' Revenge | 1.000 | n/a | n/a | 1.000 | 0.875 | recovers the true causes |
The path from this method's runner (run_game) into the bit-exact VCS substrate, and the
computation it involves. Every step links to the exact source on main. From
callstack_phaseC.json.
env_reset!(env; boot_noop_steps = 60, boot_reset_steps = 4) jutarienv_step!(env::StellaEnvironment, action::Integer) jutaribuild_shared_testbed(game; ... intervene_ram = intervene_ram!, run_intervention = run_intervention, soft_ram_peek = soft_ram_peek, ...) runnersnapshot(env::StellaEnvironment, frame::Integer) jutaribuild_variables(cand_indices_concepts) runnerinterchange(base_ckpt, tail, kind, sites, values) -> intervene_ram!/intervene_tia!; env_step! runner intervene_ram! @ das.jl:256, intervene_tia! @ :258, env_step! @ :263Scored against the exact intervention oracle Δy(u): run_intervention(checkpoint, actions, target_frame, horizon, cause).
The figure is generated from the committed record by
docs/gen_method_figures.py; the game frame
and each RAM cell's screen footprint are produced by
render_scenes.jl /
cell_footprints.jl.