Breast-CT leaderboard
128-view 2-D fan-beam sparse-view synthetic phantoms (Sidky-group breast
model, real μ range up to ~0.5). All metrics through
ddssl_ldct.metrics.evaluate_calibrated:
two-point linear intensity calibration on the foreground inside an
inscribed-circle FoV mask, then PSNR/SSIM/RMSE on the calibrated
prediction.
Baseline FBP: SSIM = 0.957, PSNR = 39.74 dB, hr = 0.
hr = max(0, 1 − rmse / baseline_rmse).
Leaderboard — every solver (rendered from the registry)
The table below is rendered live from the registry
(docs/runs/index/leaderboard.json, built by
scripts/build_registry.py from each iter’s
immutable observation.json). It lists every solver exercised on breast-CT,
ranked by val headroom (SSIM tiebreak). Breast-CT has no held-out test
set, so every column is on the val set — SSIM / PSNR / RMSE are shown as
mean ± std over the scored slices. The below-baseline / discarded solvers
(hr = 0, structurally bounded) stay on the board dimmed and unranked so the
full inventory always shows — never a top-N. No number on this page is typed by
hand. TPE runs that predate the PSNR/RMSE/time/params logging show — for
those columns (only SSIM + headroom were recorded then, and the raw recon is not
retained). params (M) is trainable parameters in millions (a raw integer for
the handful-of-params solvers); 0.000/small values are exact.
All metrics use ddssl_ldct.metrics.evaluate_calibrated:
two-point linear intensity calibration on the foreground inside an
inscribed-circle FoV mask, then PSNR/SSIM/RMSE on the calibrated prediction.
Baseline FBP: SSIM = 0.957, PSNR = 39.74 dB, hr = 0.
The dimmed (hr = 0) rows are not “just under the threshold” — they are
structurally bounded by FBP under the calibrated metric, as detailed below.
Why these fail structurally
- Self-supervised dual-domain (N2I): Noise2Inverse rewards smoothing
in the dense-view regime; the half-set FBP target carries noise the
optimiser tries to match. The DD-BF/DD-UNet supervised L2 twins above
show what fixing the loss alone gets you (
hr= 0.26 / 0.84). - Per-scene neural-implicit (NAF / R²-Gaussian): designed for
sparse-view CBCT; can’t compete with a properly-tuned FBP at 128
views on this dataset. Two retry rounds confirmed this empirically
for R²-G (2026-06-03 agentic): extending
gs_n_iterfrom [300, 800] to [10k, 40k] left SSIM at 0.89 (still hr=0); FBP-warm-starting the Gaussian positions also left SSIM at 0.89. The basis is too sparse to represent dense soft tissue at the resolution that clears baseline FBP; the inductive bias is fundamentally for sparse-view scans where FBP is weak. See findings.md 2026-06-03 entry. - TV-iterative supervised L2 (unrolled): FBP init + smooth-TV gradient + supervised L2 → the first GD step learns to do nothing; structural ceiling = baseline FBP.
- Diffusion-recon with breast-DDPM checkpoints: three checkpoint arches retrained, all hr=0 across ≥40 trials each. v1 (ch=32, 25 ep) → SSIM ~0.46; v2 (ch=64, 80 ep, val_eps_loss=0.0050) → SSIM 0.30–0.49; v3 (ch=128, 60 ep, val_eps_loss=0.0020) → SSIM 0.33–0.48 (first 5 iters). The training metric improves monotonically (loss 0.0050 → 0.0020) but the posterior-sampling reconstructions stay in the same SSIM band — the failure is the prior class, not capacity or training duration. SmallDDPM generates individually plausible breast images but they are not conditionally faithful to the input sino under DPS/DC sampling. Closing this path requires a structurally different prior (score-SDE / EDM / U-ViT) — not in scope for current solver code. Full diagnosis in findings.md 2026-06-03 entry.
Inventory-gap + Wu-TPE closure log (2026-06-09)
ItNet v1 was dispatched + Wu 2015 trainable got a full TPE pass on breast-CT on 2026-06-09:
| Solver | TPE job | Result | Status |
|---|---|---|---|
ItNet v1 (solver_itnet.py) |
762956 | hr=0.1703 at iter-20 winner (k=8, c=8, pretrain_ep=5, lr=1e-3, α=0.015) | ✅ ABOVE BASELINE — slots in at rank 15. Surprises the Mayo verdict (hr=0): on breast-CT’s broader synthetic-anatomy substrate, v1’s deeper finetune-pass lets it clear baseline. Confirms ItNet family transfer pattern: v1 < v2 < v3 on breast-CT (0.1703 → 0.5386 → 0.7342). |
Wu 2015 trainable (solver_wu_2015_trainable.py) |
762955 | COMPLETE 20/20: final hr=0.3170 (winner cluster across iter-8/12/13/15/16 at hr=0.31-0.32, all with n_bands=6, n_outer=2, range=8, window=2, ep~12-13, lr~1e-4, λ_neg~0.7) | 🎯 +45% over agentic 0.2189! TPE found a higher-n_bands + lower-lr corner the agentic search missed. FINAL at rank 10. Agentic-seed row stays at rank 14. |
Cross-dataset coverage compare (after 2026-06-09 gap-closure dispatches):
- Demo-DL (Sidky synthetic): 19 above + 0 below = 19/19 inventory variants (ItNet v1 just closed at rank 4; TV-iter supervised running iter-7, all hr=0 — confirming structural verdict)
- Breast-CT (Sidky synthetic with anatomy): 15 above + 13 below = 28 entries, full 19-inventory coverage (ItNet v1 closed; Wu trainable TPE lifted +43%)
- Mayo-LDCT (real helical, 2304-view): 12 above + 10 below + 1 deprioritised = 23 entries, full 19-inventory coverage
Methodology
See solver_plan.md. One row per solver family;
“Variant” picks the best config across all autoresearch + TPE iterations.