Grad×Input / DeepLIFT

What it does?

Grad times Input, in the DeepLIFT family, sharpens plain saliency by multiplying the gradient at each input by the input's own value. The product answers a slightly different question than the bare gradient: not just how sensitive the output is to an input, but how much that input, at its actual value, contributes to the output. DeepLIFT frames this as a difference from a reference, and the multiplication gives the method a completeness-like property, so the attributions sum to the change in the output. For images this tends to produce cleaner, less diffuse maps than raw saliency. We apply it to the VCS by multiplying each cause's gradient by its recorded value and correlating the result with the true causal map. On outputs that flow smoothly into the picture, the input factor concentrates the attribution onto the genuine causal byte and improves on plain saliency. On a sprite's position it inherits the same wall as every gradient method: the position is a discrete step set by strobe timing, so the underlying gradient is zero and multiplying by the input cannot create signal where there is none. The method is popular because it keeps the speed of a gradient while adding a contribution interpretation, so it is a fair, stronger member of the gradient family to test.

Top row (image domain, as in Paper 1): the game frame, then the oracle's true causal region and this method's attributed region — each painted onto the frame through the screen footprint of the RAM cells it implicates (brighter = more important). A faithful method's heat matches the oracle's. Bottom: per-cell importance — oracle (green) vs method (blue) — and the deletion/insertion faithfulness curves (perturb the ranked causes and watch the output move). Note: the image-domain overlay footprints are illustrative, computed on the pre-redesign boot frame; the bars, curves and all reported numbers come from the re-run records on the shared gameplay states. deeplift_pearson_corr_with_oracle = 0.822 — this example only (pong, state f90+15); the audit aggregate is below.

Reading this example's causal region

This example explains the score (score@RAM $31). Its strongest true-cause is: RAM $31 (17% of its footprint sits in the score band, the rest in the play area). Because it explains the score, its only true-cause is RAM $31, so the region covers the score and the ball but not the paddle: the paddle cell (RAM $36) is not causal for the score, so it is legitimately absent.

How it's scored

The score is the Pearson correlation of the Grad-times-Input, DeepLIFT-style attribution with the oracle's exact causal map, reported raw. Faithfulness for this method is that correlation, and the grading is always against the intervention oracle rather than another method. Multiplying the gradient by the input concentrates the attribution onto the genuine causal byte, so on smooth content outputs it improves on plain saliency and scores well. The hard case is the same as for the whole gradient family. A sprite's position is a discrete step set by strobe timing, so the naive gradient there is provably zero, and multiplying a zero gradient by the input is still zero. The audit turns on the differentiable sampler to give the method a fair chance on position, but the restored gradient is faithful on only a minority of games, so the position score stays low while the content score stays high. The completeness property, that attributions sum to the output change, does not rescue the position regime. The audit box reports the all-regime faithfulness, which averages the strong content result and the weak position result across the 42 scored games, so the single number sits between the two.

The score is measured against the §1 intervention oracle — never against another interpretability method. F (faithful) is always vs the oracle; see the execution stack. How each of F / S / M is actually computed for this method (and whether it matches the paper) is in the box just below; the numbers are in the In the audit box under it.

How F, S, M are computed here

The exact formula this method uses for each score, read from its runner, and whether it matches the paper's §3 (F ∧ S ∧ M triad) definition. From the committed audit fsm_math_phaseB.json.

Axis	Formula (as computed)	What it measures	Matches §3?
F faithfulness	\(F = \rho_{\mathrm{Pearson}}\!\left(\mathrm{attr},\ |\Delta y(u)|\right)\)	Raw Pearson correlation between the method's per-cause attribution magnitudes and the oracle's true absolute causal effects |Delta_y(u)|, scored separately for the content and position output regimes.	✓ matches gradxinput.jl: pr = pearson(attr, cmap.abs_delta); metric_name=deeplift_pearson_corr_with_oracle. Record: out/gradxinput_deeplift_air_raid_content.json (value=0.90). Scorer pilot_ig_vs_oracle.jl pearson() (raw Statistics.cor, zero-variance->0); F = pearson(attr, oracle_abs_delta) where oracle_abs_delta = abs.(cmap.delta) (true |Delta_y| per cause). Also reports spearman, precision@k, deletion/insertion AUC (paper's auxiliary F metrics). Per-regime split: separate *_content.json and *_position.json records, each carrying its own extra.triad.{F,S,M}. how it's measured — call stack: per-cause DeepLIFT grad×input attribution — gradxinput.jl:478 raw Pearson of attribution vs oracle |Δy(u)| — pilot_ig_vs_oracle.jl:173 assemble the F∧S∧M triad record — gradxinput.jl:895 true causal effect Δy(u) by bit-exact re-run — oracle_intervene.jl:242
S sufficiency	\(S = \dfrac{\#\{\,u_{\mathrm{held}}:\ |\hat{y}-y|\le\varepsilon\,\}}{|\mathrm{held\text{-}out}|}\in[0,1]\)	Fit Delta_y = a*attr + b on a calibration half of the do(u) causes, predict the disjoint held-out half, and report the fraction of held-out causes whose predicted output lands within an epsilon band of the oracle's bit-exact re-run.	✓ matches common/triad_sm.jl sufficiency_score() (interleaved calib/held split by seed, least-squares 1-D fit Delta_y~a*attr+b on calib, epsilon = max(0.5, 0.10*heldout_spread)); assembled in triad_extra_dict(). The new §3 (sec:triad) admits the fraction-within-tolerance [0,1] special case of the held-out predictive test, which this held-out do(u) fit-then-predict estimator satisfies, so it matches. (Detail: epsilon is a self-scaled band 0.10*spread; the [-1,1] correlation form is available but this method reports the [0,1] hit fraction.) how it's measured — call stack: sufficiency_score: fit Δy=a·attr+b on a calib half, predict held-out within ε — triad_sm.jl:111 assemble the F∧S∧M triad record (calls sufficiency_score) — triad_sm.jl:173 true causal effect Δy(u) by bit-exact re-run — oracle_intervene.jl:242
M minimality	\(M = |U^{\star}| / |\hat{U}| \in (0,1]\quad U^{\star}=\{u:\Delta y(u)>0\},\ \hat{U}=\{u:|\mathrm{attr}(u)|>\tau\}\)	Ratio of the number of oracle causal movers (causes with nonzero true Delta_y) to the number of cells the method names above 1e-6 of its own max attribution; null when the oracle finds no mover or the method names nothing.	✓ matches common/triad_sm.jl minimality_score() with name_frac=1e-6, mover_floor=0.0; the above-threshold named set is used (topk kept but not passed), see triad_extra_dict(). This is the paper's M = |U*|/|U_hat| in (0,1] (standardized everywhere), so it matches the new §3. (Detail: |U*| is taken as all oracle movers with Delta_y>0, an upper bound on the strictly-smallest reproducing subset, so M can be optimistic.) how it's measured — call stack: minimality_score: |U*| oracle movers / |U_hat| named cells — triad_sm.jl:57 assemble the triad record (calls minimality_score) — triad_sm.jl:178 true causal effect Δy(u) by bit-exact re-run — oracle_intervene.jl:242

✓ matches = the same quantity as §3; ◐ partial = the same kind of estimator but a differing detail; ✗ does not match = a different quantity (see the note). Definitions: F = agreement with the oracle's true causal effects Δy(u); S = held-out predictive score in [−1, 1]; M = |U*|/|Û| (true-minimal-set size / named-set size).

In the audit

This is the method's entry in the actual cross-method audit — scored on the paper's correctness triad, each axis a mean over all 42 scored games (84 committed §R records), not the single example shown above. Tradition: gradient. The example figure (Pong) is one of those records.

F faithfulness (scored vs the oracle for every method) · S sufficiency (held-out predictive score in [−1, 1]; a negative value means the explanation predicts held-out interventions worse than the unperturbed baseline; reported for the predictive methods across all three phases where the calibration/held-out split is defined) · M minimality (true-minimal-set / named-set; where the method names a cause set) — n/a otherwise, per the paper's F ∧ S ∧ M triad.

Source: leaderboard.json · the whole leaderboard is on the methods page and the Paper 2 audit.

Results per game

This method's faithfulness on each of the 42 scored games (all-regime F, and the content vs position split). Click a header to sort. Every number is read from site_data.json (methods.gradxinput.per_game).

Game	F (all)	content-F	position-F	S	M	Note
Beam Rider	0.881	0.766	0.997	0.833	1.000	recovers the true causes
Krull	0.804	1.000	0.608	0.538	1.000
Seaquest	0.762	0.813	0.710	0.958	1.000
Gopher	0.761	0.776	0.746	0.929	1.000
Pacman	0.642	0.803	0.481	0.611	1.000
Bowling	0.628	0.908	0.348	0.905	1.000
Ice Hockey	0.576	0.964	0.188	0.545	1.000
Pong	0.550	0.822	0.278	0.500	1.000
Video Pinball	0.542	0.923	0.160	0.500	1.000
Centipede	0.500	1.000	0.000	0.550	1.000	position gradient vanishes
Space Invaders	0.500	1.000	0.000	0.528	1.000	position gradient vanishes
Tennis	0.500	1.000	0.000	0.531	1.000	position gradient vanishes
Kangaroo	0.500	0.999	0.000	0.500	1.000	position gradient vanishes
Q*bert	0.500	0.999	0.000	0.500	1.000	position gradient vanishes
Chopper Command	0.499	0.998	0.000	0.952	1.000	position gradient vanishes
Double Dunk	0.497	0.994	0.000	0.808	1.000	position gradient vanishes
Pitfall	0.492	0.985	0.000	0.912	1.000	position gradient vanishes
Road Runner	0.491	0.982	0.000	0.526	1.000	position gradient vanishes
Private Eye	0.487	0.970	0.003	0.960	1.000	position gradient vanishes
Boxing	0.487	0.817	0.156	0.542	1.000
Demon Attack	0.477	0.955	0.000	0.875	1.000	position gradient vanishes
Assault	0.475	0.950	0.000	0.952	1.000	position gradient vanishes
Hero	0.474	0.947	0.000	0.889	1.000	position gradient vanishes
Carnival	0.469	0.938	0.000	0.526	1.000	position gradient vanishes
Bank Heist	0.455	0.911	0.000	0.833	1.000	position gradient vanishes
Phoenix	0.455	0.909	0.000	0.577	1.000	position gradient vanishes
Air Raid	0.448	0.896	0.000	0.464	1.000	position gradient vanishes
Jamesbond	0.447	0.865	0.028	0.893	1.000
Alien	0.430	0.860	0.000	0.500	1.000	position gradient vanishes
Riverraid	0.411	0.823	0.000	0.478	1.000	position gradient vanishes
Ms. Pac-Man	0.411	0.822	0.000	0.500	1.000	position gradient vanishes
Name This Game	0.407	0.815	0.000	0.417	1.000	position gradient vanishes
Kung-Fu Master	0.402	0.805	0.000	0.800	1.000	position gradient vanishes
Freeway	0.398	0.747	0.049	0.500	1.000
Fishing Derby	0.384	0.769	0.000	0.450	1.000	position gradient vanishes
Breakout	0.375	0.750	0.000	0.600	1.000	position gradient vanishes
Atlantis	0.365	0.730	0.000	0.450	1.000	position gradient vanishes
Montezuma's Revenge	0.362	0.724	0.000	0.438	1.000	position gradient vanishes
Venture	0.349	0.698	0.000	0.474	0.750	position gradient vanishes
Berzerk	0.287	0.555	0.020	0.444	1.000	position gradient vanishes
Frostbite	0.272	0.484	0.059	0.150	1.000
Yars' Revenge	0.247	0.493	0.000	0.429	1.000	position gradient vanishes

Call stack — how it runs on jutari / jaxtari

The path from this method's runner (main) into the bit-exact VCS substrate, and the computation it involves. Every step links to the exact source on main. From callstack_phaseB.json.

Applied on the substrate

1. env reset + per-frame step of the bit-exact VCS — env_reset! / env_step! jutari
2. shared gameplay-state testbed — build_shared_testbed jutari

The computation (differentiable / gradient path)

1. CONTENT output y = one-hot RAM read at score cell (forward-exact, Theorem 1) — content_output -> soft_ram_peek jutari-diff call site tools/xai_study/phaseB_attribution/gradxinput.jl:268
2. dy/du over the 128-byte RAM tape (one-hot content gradient e_s) — content_grad_over_ram runner
3. autodiff of the content read — Zygote.gradient(r -> content_output(r, score_idx), ...) Zygote
4. attribution maps: GradxInput u*dy/du, DeepLIFT (u-baseline)*dy/du (completeness-exact on the linear content unit), saliency |dy/du| — attributions_over_ram runner
5. POSITION/pixel output is a discrete argmax of sprite column => differentiable RAM handle identically zero (vanishing gradient, modeled honestly) — position_grad_over_ram runner
6. POSITION path restored via bilinear sampler (shared testbed) — _st_sampler_position_read runner-diff

Scored against the exact intervention oracle Δy(u): run_intervention.

On jaxtari (JAX / GPU): the same differentiable path has a bit-exact JAX sibling (Theorem 1, forward-exact) for batched GPU runs — JAX-equivalent differentiable path (bit-exact per Theorem 1, for GPU batching): jaxtari/jaxtari/diff/soft_step.py, soft_mem.py, straight_through.py

Implementation

tools/xai_study/phaseB_attribution/gradxinput.jl

Reference

Shrikumar et al. 2017

Record

gradxinput_deeplift_pong_content.json

All records

phaseB_attribution/out

The figure is generated from the committed record by docs/gen_method_figures.py; the game frame and each RAM cell's screen footprint are produced by render_scenes.jl / cell_footprints.jl.