Demo-DL leaderboard

128-view 2-D fan-beam sparse-view synthetic phantoms (Sidky-style random ellipse phantoms — the “demo” track used as a fast iteration substrate). Smaller and faster than the breast-CT track; useful for prototyping.

All metrics through ddssl_ldct.metrics.evaluate_calibrated: two-point linear intensity calibration on the foreground inside an inscribed-circle FoV mask, then PSNR/SSIM/RMSE on the calibrated prediction. hr = max(0, 1 − rmse / baseline_rmse) where the baseline is the calibrated FBP (demo-intensity-calibrated-tpe-* family).

The 2026-05-19 calibrated-scoring convention is the canonical metric; earlier uncalibrated runs (slug prefix demo-dl-*, dl-sparse-view-*) are kept at the bottom for historical context and are not directly comparable.

Leaderboard — every solver (rendered from the registry)

The table below is rendered live from the registry (docs/runs/index/leaderboard.json, built by scripts/build_registry.py from each iter’s immutable observation.json). It lists every solver exercised on the calibrated demo-DL substrate, ranked by val headroom (SSIM tiebreak). Demo-DL has no held-out test set, so every column is on the val set — SSIM / PSNR / RMSE are shown as mean ± std over the scored slices. Any below-baseline / discarded solver stays on the board dimmed and unranked so the inventory is always complete — never a top-N. No number on this page is typed by hand. The pre-2026-06 TPE runs show — for PSNR/RMSE/time (only SSIM

• headroom were recorded then). params (M) is trainable parameters in millions.

The registry scope is the calibrated convention only (demo-intensity-calibrated-tpe-* + the demo-dl-claude-agentic-* N2I seeds). The legacy uncalibrated demo-dl-* runs use a different scoring rule and are not in the registry; the few kept for historical context are in the “Earlier uncalibrated runs” section near the bottom.

Below-baseline inventory (hr = 0, structural STOPs)

None of the 18 variants tested on demo-DL fell below baseline. The synthetic Sidky-style 128-view ellipse substrate is broad enough that every learned and non-learned solver finds some working corner.

Inventory-gap closure log (2026-06-09)

Two solvers from the solver_plan.md 19-solver inventory were dispatched on demo-DL on 2026-06-09 to close the coverage gap:

Cross-dataset coverage compare (gap closure 2026-06-09 — FINAL):

All three datasets now have full 19-solver inventory coverage. The gap closures revealed two surprises: (a) ItNet v1 clears baseline on both synthetic datasets (despite Mayo verdict hr=0) — the v1 architecture is competitive with v3 on demo-DL (0.4665 ≈ 0.4676); (b) Wu trainable on breast-CT lifted +45% from agentic (0.2189 → 0.3170) via TPE finding a higher-n_bands + lower-lr corner the agentic search missed.

The demo-DL “every solver works” pattern is a useful sanity check for the substrate but a poor predictor of behaviour on the two realistic-data benchmarks — see solver_plan.md Step 1 for the cross-dataset transfer record.

Constrained vs. unconstrained DDPM (Step 4 of solver_plan.md)

The Demo-DL DDPM was trained two ways:

• ddpm_constrained_final.pt — train_n=200 (the same 200 phantoms the supervised solvers train on; no test-set distribution leakage).
• ddpm_unconstrained_final.pt — train_n=2000 (different seed range from training/test, larger sample → richer prior).

The unconstrained variant scored hr=0.4530 vs constrained’s 0.4418, +0.0112 hr from “seeing more (random-seed-disjoint) ellipse phantoms”. The gap is the empirical answer to “how much does the DDPM prior benefit from a larger / test-distribution-overlapping training set?” for this benchmark — modest but real.

Earlier uncalibrated runs (not directly comparable)

Slug prefix demo-dl-* and dl-sparse-view-* (pre-2026-05-19 convention). Kept for historical context; the reported hr comes from a different scoring rule and is systematically higher than the calibrated equivalent. Top entries only:

Methodology

See solver_plan.md for the full benchmark protocol — dataset construction, baseline FBP definition, calibrated metric, and per-solver hyperparameter spaces.