The infrastructure a skeptical reviewer can use to confirm the code and experiments are real: an external oracle, conformance harnesses, a large test suite, continuous integration, and an open development log.
The strongest defence against “the AI hallucinated it” is that the two differentiable ports are checked against xitari — a separately written, pre-existing C++ Atari 2600 emulator that is not part of this project. On all 64 ALE-supported games the ports reproduce xitari’s 128 B of RAM byte-for-byte and its 210×160 framebuffer pixel-for-pixel. Hallucinated code does not accidentally match an external reference to the bit. The reference emulator lives at xitari/ with its determinism patch at tools/xitari_conformance_seed.patch.
For a line-by-line walk of each harness's call stack, see the conformance code tour.
| Harness | What it asserts | Source |
|---|---|---|
| PXC1 | Replay an xitari JSONL trace against a port; assert per-frame RAM byte-identity. | jaxtari/tests/test_pxc1_conformance.py |
| PXC2 | Drive jaxtari live + load a jutari fixture trace; assert the two ports diverge from xitari identically. | jaxtari/tests/test_pxc2_jaxtari_vs_jutari.py |
| PXC-S | Per-frame 210×160 framebuffer diff, xitari vs both ports (~23 min). | jaxtari/tests/test_screen_conformance.py |
| PXC4 | Klaus Dormann 6502 functional test on flat 64K memory (CPU-core validator). | jaxtari/tests/test_pxc4_klaus_dormann.py |
| Tool | Purpose | Source |
|---|---|---|
| trace_dump | C++ driver over xitari — emits deterministic JSONL frame traces (RAM, optional screen + CPU state). | tools/trace_dump.cpp |
| check_trace | Replay a trace against a port and diff byte-by-byte (Python + Julia). | tools/check_trace.py |
| ram_diff | Per-frame RAM divergence, jutari vs xitari. | tools/jutari_xitari_ram_diff.py |
| conformance seed patch | Pins xitari’s RNG seed for reproducible replay. | tools/xitari_conformance_seed.patch |
| Port | Count | Location |
|---|---|---|
| jaxtari (pytest) | 837 def test_* functions across 60 files | jaxtari/tests/ |
| jutari (Julia) | 1,215 @test assertions | jutari/test/runtests.jl |
| Workflow | Runs | Source |
|---|---|---|
| test.yml | Fast PR gate on every push/PR: full jaxtari pytest + jutari Pkg.test() + PXC2 cross-check. | .github/workflows/test.yml |
| heavy.yml | Nightly (04:00 UTC) + manual: JAX-autodiff and slow ROM/screen groups on fresh runners. | .github/workflows/heavy.yml |
| Document | Role | Source |
|---|---|---|
| bug_fix_log.md | 7,525 lines — newest-on-top history of every bug, patch and ruled-out hypothesis. | bug_fix_log.md |
| STATUS.md | Per-phase commit/test/deferral ledger. | STATUS.md |
| RESULTS.md | Milestone write-up + reproducibility section. | RESULTS.md |
jutari conformance (~20 s)
cd jutari && julia --project=. -e 'using Pkg; Pkg.test()'jaxtari full suite (long)
cd jaxtari && .venv/bin/python -m pytest -qscreen conformance (~23 min)
jaxtari/.venv/bin/pytest jaxtari/tests/test_screen_conformance.pyregenerate an xitari reference trace
./tools/trace_dump --rom xitari/roms/pong.bin --actions tools/fixtures/actions/pong_noop_10.txt > /tmp/pong.jsonl