The verification ladder
Nothing of the original game runs inside the port, so every claim that it plays like retail rests on a check someone can re-run. No single check covers the whole claim. The project stacks six, each answering one question and each blind to something the next one sees. This page puts them in order, says what each holds retail to and what gates it, and names where each one stops.
The ladder
Coverage. 765 of the 1043 retail addresses a // PORT: tag names (73.3%) are run by a differential; 554 tests run 783 retail routines in all. Generated from the tests' declarations by scripts/ci/update-progress-metrics.py, and ratcheted by differential-coverage.py --check.
| Rung | Blind to | Seen by |
|---|---|---|
| Format docs | What the code does with the data | Provenance names the routine; the differential runs it |
| Port provenance | The arithmetic under a correctly named tag | The differential |
| Kernel differential | When, and with what, the engine calls the kernel; any routine no test runs | Stills and films; the --no-differential worklist |
| Retail-compare stills | Time: order, length and timing around the instant | Films |
| Films | Everything outside the recorded stretch | The full-game ladder |
| Full-game ladder | Fidelity: it compares nothing with retail | Every rung below it |
1 · Format docs
Every format the engine reads has a byte-level page whose layout is traced to the routine that reads it, with a confidence level, and a parser the extraction pipeline and the patcher share (formats). The denominators come from the disc: disc coverage counts code bytes inside dumped functions and entries with a known format class, and byte accounting counts the bytes a parser actually consumes. The patcher's disc-gated oracles re-decode a patched copy, so a writer is held to its reader.
Blind to what the code does with the data: a correctly parsed table can still be indexed, clamped or summed differently.
2 · Port provenance
A port is a Rust function tagged // PORT: FUN_<addr>. The port catalog derives what is ported and what is reachable from those tags; port provenance asks whether the address names the routine the code implements. It ranks suspicious pairings by signals read from the disassembly, the docs and the disc images: a routine that shares nothing with the rest of its module, one given a defining description on two unrelated pages, a tag citing evidence its routine's dump does not carry, a tag on an address the repository elsewhere says starts no routine. check-port-provenance.py --check fails a commit that adds an unwaived finding, and each waiver says what the bytes showed.
Blind to the arithmetic: a tag can name exactly the right routine while the Rust under it computes something else.
3 · The retail kernel differential
The retail kernel differential runs the retail routine off your disc in crates/retail-exec, a user-mode R3000 interpreter with a GTE written from the hardware documentation, beside the Rust kernel that ports it. Both sides take the same generated inputs, caps and empty cases included, and one scripted rand() stream; the whole regions they write are compared, not chosen fields.
- Declared.Each test opens with
differential!(scus(VA))ordifferential!(prot(ENTRY, VA)), naming every routine it compares. - Proven.The interpreter logs every entry it runs with the bytes there; the test fails unless each declared routine was entered and its bytes are the named image's. The wrong address, or the right one in the wrong overlay, fails.
- Counted.
differential-coverage.py --checkreads the declarations with no disc, fails a test with none, and ratchets the covered count, the routine count and the test count. The share is on the landing page.
Blind to when and with what the engine calls the kernel: a routine that matches here can still be wired a frame late, fed the wrong record, or never called. A routine no test runs is not checked at all; port-catalog.py --no-differential --live lists the live ones.
4 · Retail-compare stills
The retail comparison corpus seats the engine where each retail save state stood, through the engine's own load path, and scores the frame and the observables the state's RAM holds: scene, camera, player, placements, music, flags, party, enemies. Per-state, per-channel scores may only rise, and a fall needs a ledger entry. Capture coverage takes its denominator from the disc, so the corpus widens where no state yet looks.
Blind to time: a state is one instant, and a cast's order, length and timing, or a walk's path between two states, are invisible to it.
5 · Films
The cast-sequence oracle records one cast on both sides and pairs frames on the game's own clock, the action state and then the cast module's phase; the walk film does the same for a field walk under a fixed pad schedule. Recordings carry retail pixels and stay off the repository, so a film has no committed baseline: a change reports its before and after on the same command.
Blind to everything outside the recorded stretch, and to residuals smaller than two retail recordings of one cast differ from each other.
6 · The full-game ladder
The full-game ladder plays New Game to credits with no human on pad input, segment by segment from retail saves at each story milestone and as one continuous run. Its tiers are ratcheted, and a seed set repeats the run on other random streams.
Blind to fidelity: it proves the game can be finished in the port and compares nothing with retail, which is what the rungs below it do.
Why the differential is the hinge
The lower two rungs are about naming: what the bytes are, and which routine a Rust function claims to be. The upper three are about behaviour as a player sees it, sampled where a capture happened to stand. The differential is the first rung that runs retail's own code, and the only one that reaches inputs no capture holds. It also settles the rung below it: the provenance checker reports a differential-run address as verified, because the declaration's run proves which bytes executed.
Its findings show the gap it closes. Ports that read right against their dumps, carried a correct tag and passed their own unit tests still computed something else: a level-up applier that drew none of retail's rand() calls, an arctan table rounded where retail's truncates, a stat aggregator whose grant scan had no port behind its heading, a status tick that read the tracker where retail reads the word. What it has found lists them routine by routine.
What a gate does and does not mean
Each gate fails on its own and is committed as a ratchet that may only improve, and a green gate means only what that gate measures. A disc-gated test that skips for want of its data is not a pass, and an oracle whose retail side is the engine's own model cannot disagree with it. The method written engine-neutral for another project is the retail-oracle method; the seven porting steps these gates sit under are porting by reimplementation; the four instruments side by side are measuring against retail.