Agent4CT

Agent4CT is a continuously-running LLM agent that improves a CT reconstruction codebase by editing it, running short experiments on a Slurm GPU cluster, and keeping or discarding changes based on the resulting metrics. The pattern is borrowed from karpathy/autoresearch and generalised to five CT-imaging benchmarks — the Pentathlon.

The Pentathlon

Five public AAPM CT reconstruction challenges, run on the same recon backbone (PYRO-NN fan-beam, Wagner / Siemens-AS geometry):

See Pentathlon for the headroom-recovered scoring and per-challenge train/val/test splits.

How the loop runs

Each iteration is one 5-minute Slurm job. The agent edits pentathlon/<challenge>/solver.py, the harness runs the job, computes the validation metric, and decides keep / discard via git. Every 30 iterations a 1-hour stage job runs on a 3× larger subset to catch overfitting. Test sets are touched exactly once, at the end. See Agents.

🏆 Leaderboards

Best-of-best per solver per dataset under the canonical calibrated-SSIM-headroom scoring convention.

The champion per dataset (single canonical ranking = headroom, SSIM tiebreak) is rendered live from the registry below, so it never drifts from the dataset boards.

See Leaderboards for the full per-solver rankings (every solver, below-baseline dimmed, with comparison images) and the cross-dataset summary.

Quick links

• 🏆 Leaderboards — calibrated headroom rankings per dataset
• 📊 Live dashboard — every run, every iteration, scratch pad with images
• 🧪 Setup — env, cluster, data
• 🥇 Pentathlon — challenges + scoring
• 🧠 Agents — the autoresearch loop
• ⚡ Performance — PYRO-NN + NFS tips
• 📓 Findings — cross-cutting insights from the autoresearch loop
• 📦 GitHub