Phase A · neuroscience battery

A3 · Tuning curves

cf. Jonas & Kording 2017

← back to the method catalogue

What it does?

A tuning curve measures how strongly a unit responds to some variable. In neuroscience one plots a neuron's firing rate against, say, the angle of a bar, and a sharp peak is read as the neuron being 'tuned' to that feature. We build the same curves for RAM cells. For each cell we plot its value against a game variable and against screen luminance, then measure how strongly the cell tracks each one. A cell that closely follows a game variable is called tuned to it. The intuition is that a strongly tuned cell must encode that variable and therefore drive the behaviour. This is one of the workhorses of systems neuroscience. On the VCS it is actively misleading. Many cells co-vary with the beam clock and the frame counter, so apparent tuning is cheap and common. A cell can track a game variable perfectly while playing no causal role in producing it. The method thus flags cells that look meaningful but are not, which is exactly the trap we set out to measure. It is the same present-versus-used problem that also defeats linear probing.

A3 · Tuning curves result

Each point is a RAM cell: its true causal importance (x) against its tuning strength to a game variable (y). Red points are strongly tuned yet not causal — the trap the metric measures. A3_game_variable_spurious_tuning_rate = 0.500 — this example only (pong, state gameplay(seed=0,prefix=90)+15); the audit aggregate is below.

How it's scored

The score is a spurious-tuning rate: the fraction of cells that are strongly tuned to a game variable yet are not among the oracle's causal cells. We first find the cells whose value tracks a game variable strongly, then ask the intervention oracle which of those cells actually cause the output. A tuned cell that is not causal is a false positive, and the spurious-tuning rate counts them. Faithfulness is one minus that rate, so a method that only flags true causes scores high and a method that flags many non-causal cells scores low. On the VCS the rate is high, because many cells co-vary with the beam clock, so tuning is cheap and often unrelated to causation. This is the same present-versus-used trap that also defeats linear probing: a cell can carry a variable without the program using it. The grading is always against the oracle. There is no sprite-position hard-case here, because the output is a tuning judgement, not a coordinate. Because per-game correlations for this family can go negative, the aggregate clips each per-game value at zero before averaging, so a method with no real signal lands near zero rather than at a spurious floor. The audit box reports the aggregated faithfulness.

The score is measured against the §1 intervention oracle — never against another interpretability method. F (faithful) is always vs the oracle; see the execution stack. How each of F / S / M is actually computed for this method (and whether it matches the paper) is in the box just below; the numbers are in the In the audit box under it.

How F, S, M are computed here

The exact formula this method uses for each score, read from its runner, and whether it matches the paper's §3 (F ∧ S ∧ M triad) definition. From the committed audit fsm_math_phaseA.json.

AxisFormula (as computed) What it measuresMatches §3?
F
faithfulness
\(F = \rho_{\mathrm{Spearman}}\!\left(\mathrm{attr},\ \Delta y(u)\right)\)Spearman correlation between per-cell game-variable tuning selectivity and the oracle causal importance over 8 candidate cells.✓ matches Scored vs oracle causal importance (not vs external labels); the per-game clip-at-0-then-mean is applied at aggregation (paper cites A3 = 0.213 clip-at-zero mean). Runner A3_tuning.jl,; record out/A3_tuning_pong.json triad.F=-0.007.
how it's measured — call stack:
  1. F = Spearman(tuning selectivity, oracle causal importance) — A3_tuning.jl:473
  2. score_triad: builds selectivity and scores it against the oracle — A3_tuning.jl:470
  3. true causal effect Δy(u) by bit-exact re-run — oracle_intervene.jl:242
S
sufficiency
\(S = \rho_{\mathrm{Pearson}}\!\left(\mathrm{attr},\ \Delta y_{\mathrm{held\text{-}out}}(u)\right) \in [-1,1]\)Pearson correlation of the selectivity map vs the held-out do(base+37) screen break.✓ matches Held-out predictive correlation in [-1,1] of the selectivity map vs the held-out do(base+37) screen break: the §3 (sec:triad) form. A3_tuning.jl; record S=0.441 (battery air_raid).
how it's measured — call stack:
  1. S = Pearson(selectivity, held-out do(base+37) screen break) — A3_tuning.jl:474
  2. true causal effect Δy(u) by bit-exact re-run — oracle_intervene.jl:242
M
minimality
\(M = |U^{\star}| / |\hat{U}| \in (0,1]\quad U^{\star}=\{u:\Delta y(u)>0\},\ \hat{U}=\{u:|\mathrm{attr}(u)|>\tau\}\)Number of oracle-causal cells over the number of strongly-tuned cells the method names.✓ matches Runner now emits M = |U*|/|U_hat| (M_note '|U*|=10 (oracle movers) / |U_hat|=7 (method above-threshold)'), the paper's set-size ratio. Record out/A3_tuning_*.json extra.triad.M. Replaced the former 1 − spurious-tuning rate.
how it's measured — call stack:
  1. minimality_score(selectivity, oracle importance) — A3_tuning.jl:485
  2. minimality_score: |U*| oracle movers / |U_hat| named cells — triad_sm.jl:57
  3. true causal effect Δy(u) by bit-exact re-run — oracle_intervene.jl:242

✓ matches = the same quantity as §3; ◐ partial = the same kind of estimator but a differing detail; ✗ does not match = a different quantity (see the note). Definitions: F = agreement with the oracle's true causal effects Δy(u); S = held-out predictive score in [−1, 1]; M = |U*|/|Û| (true-minimal-set size / named-set size).

In the audit

This is the method's entry in the actual cross-method audit — scored on the paper's correctness triad, each axis a mean over all 42 scored games (42 committed §R records), not the single example shown above. Tradition: correlational. The example figure (Pong) is one of those records.

0.213F — faithfulness vs oracle (mean over 42 games, ±0.080 CI95)
0.200S — sufficiency: held-out predictive (n/a where the paper does not define this axis)
0.928M — minimality: true-minimal-set / named-set (n/a otherwise)
42committed records aggregated
0.00human-plausibility proxy

F faithfulness (scored vs the oracle for every method) · S sufficiency (held-out predictive score in [−1, 1]; a negative value means the explanation predicts held-out interventions worse than the unperturbed baseline; reported for the predictive methods across all three phases where the calibration/held-out split is defined) · M minimality (true-minimal-set / named-set; where the method names a cause set) — n/a otherwise, per the paper's F ∧ S ∧ M triad.

Source: leaderboard.json · the whole leaderboard is on the methods page and the Paper 2 audit.

Results per game

This method's faithfulness on each of the 42 scored games (all-regime F; this method has no content-vs-position split (see the note below)). Click a header to sort. Every number is read from site_data.json (methods.A3_tuning.per_game).

content-F and position-F are n/a for this method. The content-vs-position split applies only to pixel-attribution methods — those that produce an attribution map over the framebuffer, whose score can be read separately on the smooth content outputs and the discrete position outputs (where the naive gradient is provably zero). This method does not attribute to a pixel output; it is graded by its own fidelity metric (A3_game_variable_spurious_tuning_rate), reported as the F column, so there is no content or position regime to separate.

M is n/a on 5 of these games. Minimality is M = |U*|/|Û|, the size of the true minimal cause set over the size of the set the method actually names. On a game where the method names nothing (|Û| = 0 — it discovers no circuit, its attribution map is all-zero, or it decodes no cell), that ratio is undefined, so M is left blank rather than scored. The aggregate M is the mean over the 37 games where the method did name a cause set.

S is negative on 13 of these games. Sufficiency asks whether the explanation can predict the outcome of held-out interventions it was not fit on; for this method S is the held-out predictive correlation between its claim and the true effect, so it lies in [−1, 1]. A value near 0 means the explanation predicts no better than the unperturbed output; a negative value means it predicts worse — its held-out claim is anti-correlated with the truth (named yet causally unused). The aggregate S is the mean over the scored games.

GameF (all)content-Fposition-FSMNote
Name This Game0.866n/an/a0.7541.000recovers the true causes
Freeway0.829n/an/a0.8731.000
Boxing0.681n/an/a0.7281.000
Double Dunk0.617n/an/a0.8081.000
Bowling0.603n/an/a0.5731.000
Bank Heist0.564n/an/a0.2471.000
Berzerk0.513n/an/a0.2230.875
Fishing Derby0.491n/an/a0.4900.714
Frostbite0.476n/an/a0.3461.000
Q*bert0.467n/an/a0.1830.467
Air Raid0.445n/an/a0.4411.000
Demon Attack0.406n/an/a0.2000.750
Yars' Revenge0.395n/an/a0.2230.700
Krull0.305n/an/a0.1841.000
Tennis0.231n/an/a-0.2171.000
Hero0.229n/an/a-0.1141.000
Private Eye0.223n/an/a0.3271.000
Jamesbond0.188n/an/a0.1300.875
Pitfall0.146n/an/a0.2580.846
Alien0.105n/an/a0.2451.000
Seaquest0.078n/an/a0.3111.000
Assault0.051n/an/a-0.0040.900
Phoenix0.025n/an/a-0.1841.000no true-cause signal
Chopper Command0.007n/an/a0.1800.769no true-cause signal
Atlantis0.000n/an/a-0.4461.000no true-cause signal
Beam Rider0.000n/an/a0.000n/ano true-cause signal
Breakout0.000n/an/a-0.3811.000no true-cause signal
Carnival0.000n/an/a-0.0281.000no true-cause signal
Centipede0.000n/an/a-0.5891.000no true-cause signal
Gopher0.000n/an/a0.000n/ano true-cause signal
Ice Hockey0.000n/an/a0.0321.000no true-cause signal
Kangaroo0.000n/an/a0.000n/ano true-cause signal
Kung-Fu Master0.000n/an/a0.000n/ano true-cause signal
Montezuma's Revenge0.000n/an/a-0.2261.000no true-cause signal
Ms. Pac-Man0.000n/an/a0.000n/ano true-cause signal
Pacman0.000n/an/a-0.3041.000no true-cause signal
Pong0.000n/an/a-0.4121.000no true-cause signal
Riverraid0.000n/an/a0.0921.000no true-cause signal
Road Runner0.000n/an/a0.0830.429no true-cause signal
Space Invaders0.000n/an/a-0.0701.000no true-cause signal
Venture0.000n/an/a-0.0831.000no true-cause signal
Video Pinball0.000n/an/a0.4721.000no true-cause signal

Call stack — how it runs on jutari / jaxtari

The path from this method's runner (compute_game) into the bit-exact VCS substrate, and the computation it involves. Every step links to the exact source on main. From callstack_phaseA.json.

Applied on the substrate

  1. assemble the shared gameplay-state testbed + cause-density gate — build_a3_shared_state(game) -> build_shared_testbed(game; ...) runner
  2. build_shared_testbed definition: booted checkpoint + shared output + gate — function build_shared_testbed(game; ...) runner
  3. cause-density gate counts causes with |Delta_y|>floor via the oracle — cause_density = count(>(floor), deltas) runner
  4. record a JutariRecord.Trajectory: step the ROM frame-by-frame and log RAM+screen per frame (the tuning-curve data matrix) — record_parity_trajectory(game, actions) runner
  5. step the real emulator one frame per action while recording — env_step!(env, Int(actions[t])) jutari
  6. read RAM into the trajectory tape each frame — get_ram(env) jutari

The computation

  1. derive game-variable signals (candidate-cell time series) from the trajectory tape — game_variable_signals(traj, cands) runner
  2. derive luminance + frame-clock reference signals — luminance_and_clock(traj) runner
  3. fit tuning curves (method under test): per-cell |Pearson correlation| selectivity to game variables / luminance / clock, self-column excluded — run_a3_tuning(traj, cands, oracle_importance; tau=0.7) runner

Scored against the exact intervention oracle Δy(u): run_intervention(checkpoint, actions, target_frame, horizon, cause).

Implementation
tools/xai_study/phaseA_kording/A3_tuning.jl
Reference
cf. Jonas & Kording 2017
Record
A3_tuning_pong.json
All records
phaseA_kording/out

The figure is generated from the committed record by docs/gen_method_figures.py; the game frame and each RAM cell's screen footprint are produced by render_scenes.jl / cell_footprints.jl.