Port catalog
The project's progress ledger, kept at function granularity. Reverse-engineering the game means walking each retail function from disassembly to documentation to a from-scratch Rust reimplementation - the port catalog measures where every function stands, derived from the tree itself rather than from a hand-maintained checklist. You only need this page if you are contributing to the reverse-engineering or engine tracks.
At a glance
- Tool
scripts/ci/port-catalog.py; drift checkerscripts/ci/check-port-tags.py- Unit
- One row per retail function address, with independent signals: dumped, documented, ported, live (opt-in), ignored
- Inputs
- Ghidra dumps under
ghidra/scripts/funcs/, citations underdocs/,// PORT:tags undercrates/ - Outputs
target/port-catalog/:catalog.csv/.md,open-work.mddashboard,live-audit.md- Gate
--checkratchets every figure againstscripts/ci/port-catalog-baseline.jsonin the pre-commit hook- Denominator
- What the project cites; disc coverage is the sibling measure denominated in the game's own bytes
What this solves
Reach for it when you are picking up work and want a real worklist: which retail functions are understood but not yet ported, which are ported but undocumented, which nobody has looked at, and which ports exist but nothing in the engine ever calls. A status table in a doc rots the day after it is written; the catalog measures each column live, so if you port a function and tag it, the next run knows.
python3 scripts/ci/port-catalog.py --dashboardThe status matrix
Each row is one retail function address. The columns are independent signals, plus an axis for scope-excluded addresses (statically-linked PsyQ SDK code - Sony's standard PSX library - which the from-scratch port maps to native equivalents rather than porting line by line).
| Column | Source of truth |
|---|---|
| dumped | A Ghidra decompiler dump exists under ghidra/scripts/funcs/ (gitignored - regenerable from the Ghidra project). |
| documented | The address is cited from at least one file under docs/ (FUN_<addr> or 0x<addr>, case-insensitive). |
| ported | A Rust source under crates/ carries a // PORT: FUN_<addr> tag for that address. |
| live | The tagged Rust symbol is reachable, through non-test code, from a host entry point. Opt-in (--live); see reachability. |
| ignored | Listed in scripts/ci/port-catalog-ignore.toml as a non-port-site (BIOS thunk, libgte, libsnd, ... - see ignore list). |
The point of the matrix is that the cross-cuts are the worklists:
| Cross-cut | Meaning | What to do |
|---|---|---|
dumped + documented, not ported, not ignored | Port worklist. Understood, not yet implemented, not PsyQ infra. | Sort by citation count to find high-leverage helpers first. |
cited but not dumped | Dump worklist. Some dump references this address but no dump exists for it. | Add to ghidra/scripts/dump_funcs.py TARGETS. |
ported but not documented | Provenance gap. A tag with no doc mentioning the source function. | Backfill the doc, or remove a wrong tag. |
ported but not dumped | Provenance gap, opposite axis. | Dump it, or question the tag. |
ported but not live | Inert port. Implemented, but no host reaches it. | Wire it, or disclose it with NOT WIRED:. |
The tag grammar
Three structured comments in Rust source carry everything the catalog knows about the engine side.
| Tag | Claim | Sets "ported"? |
|---|---|---|
// PORT: FUN_<addr> | This Rust item implements that retail function. | Yes - the only signal trusted. |
// REF: FUN_<addr> | This file cites that function (callee, cross-reference) without claiming to port it. | No; suppresses drift-checker warnings. |
NOT WIRED: (in the tag's own doc block, or //! NOT WIRED opening a module doc) | The port exists but nothing in a host reaches it yet. Read by --live-audit. | n/a - a disclosure, not a status. |
WIRED: (own doc block) | Opts one item out of a module-level NOT WIRED blanket. | n/a |
// PORT: FUN_801dd35c // single address
// PORT: FUN_801dd35c, FUN_801cf244 // multiple on one line
// PORT: FUN_801dd35c (sub-mode jump table) // trailing context allowed
//! PORT: FUN_801dd35c // inside `//!` module doc
/// PORT: FUN_801dd35c // inside `///` outer doc
Tag placement rules
- Plain mentions of
FUN_<addr>in comments are ignored - they show up in many contexts that don't imply a port and would noisily inflate the column. - Address must be lowercase hex in the SCUS / overlay code range (SCUS
0x80010000-0x8006FFFF, overlays0x801C0000-0x8020FFFF). Match is line-local. - One retail function can be ported into more than one crate (a formula shared between
engine-vm::battle_formulasand anengine-corehelper); the catalog lists every crate that tags the address. - Add the tag once, on the Rust function that implements the behaviour. Don't tag every caller.
- Prefer a
///tag on the function to a//!tag on the module. A module tag anchors the address to the whole file, which is coarse enough to report the address wired on the strength of any other function in the same file. Same for a plain data struct: tag the function that computes the value, and let the struct carry aREF:. // REF:takes the same comment shapes and multi-address syntax asPORT:.
Tag drift checker: check-port-tags.py
scripts/ci/check-port-tags.py walks crates/engine-*/src/**.rs and warns when a FUN_<addr> citation lacks a matching // PORT: or // REF: tag in the same file - the "I ported X but forgot the tag" pattern.
python3 scripts/ci/check-port-tags.py # default = --staged
python3 scripts/ci/check-port-tags.py --scan-all # full audit
python3 scripts/ci/check-port-tags.py --strict # exit 1 on warning
python3 scripts/ci/check-port-tags.py --addr 80019b28 # drill-down
python3 scripts/ci/check-port-tags.py --backfill-refs # one-shot grandfather pass
- Scope rule: only files that already carry a
// PORT:tag are checked; pure-docs files are skipped. - Backfill:
--backfill-refsrewrites in place, inserting a//! REF: ...block after the leading module doc of each port-bearing file with untagged citations. - Pre-commit: the shipped hook runs
--staged --quietaftercargo clippy, warn-only; CI can tighten with--strict.
Usage
python3 scripts/ci/port-catalog.py # global catalog -> target/port-catalog/
python3 scripts/ci/port-catalog.py --missing-ports # dumped + documented, not ported (excludes ignore-list)
python3 scripts/ci/port-catalog.py --missing-dumps # cited but not dumped
python3 scripts/ci/port-catalog.py --addr 801dd35c # drill-down on one address
python3 scripts/ci/port-catalog.py --feature title-screen # BFS from a feature's roots
python3 scripts/ci/port-catalog.py --dashboard # open-work rollup -> open-work.md
python3 scripts/ci/port-catalog.py --live # add the reachability column
python3 scripts/ci/port-catalog.py --not-live # ported but unreachable from any host root
python3 scripts/ci/port-catalog.py --live-audit # reachability vs `NOT WIRED:` disclosures
python3 scripts/ci/port-catalog.py --check --live # ratchet every baselined figure
Further flags: --md (markdown to stdout), --ported-only, --ignored-only, --include-ignored, --list-features, --dashboard-top N, --allow-uncompared, --update-baseline, --selftest. Output lands in target/port-catalog/ (gitignored).
Feature views and the dashboard
A feature is a named set of seed addresses (roots) in scripts/ci/features.toml, plus optional stop_at boundaries and a depth cap. --feature <name> filters the catalog to addresses reachable from the roots via the citation graph (one edge per "this dump cites that address"). Use it to find unported helpers in scope of one feature (--feature X --missing-ports) or to spot shared-infrastructure spillover that wants a stop_at entry. The graph only has edges between dumped functions, so the frontier widens as dumps land - start tight, widen as you dig in.
--dashboard emits open-work.md, a single regenerable page: global counts, a per-feature status table, each feature's top-N missing ports sorted by citation count (high-leverage first), the ignore-list summary, and any provenance gaps. The question-level companion - open hunts rather than per-function status - is open RE threads.
Reachability: the live axis
A // PORT: tag records that a Rust function implements a retail one; it says nothing about whether anything ever calls that Rust function. --live adds the missing column by building a call graph over every Rust file in the workspace and asking, per tag, whether its symbol is reachable from a host entry point. The pass is markedly slower and stays opt-in.
| Root family | What it covers |
|---|---|
fn main in a [[bin]] target | Every CLI subcommand, plus each GUI binary's dispatch and window-loop setup. |
#[wasm_bindgen] exports | The browser's entry points into this site's viewer, play and patcher pages. |
impl ApplicationHandler for T methods | The per-frame native GUI surface - winit calls these back from outside the tree. The trait set is EXTERNAL_DISPATCH_TRAITS. |
Nothing else is a root: a pub fn no host reaches is exactly the inert-port case the axis exists to find. Test code is excluded on purpose - "called only by a unit test" is precisely the condition a NOT WIRED: tag reports.
The graph resolves calls by name, not by type, and every ambiguity resolves toward reachability. The asymmetry is what makes the axis usable: false positives on live are expected (method-name collisions like .tick() link callers to every same-named method), false negatives are rare. So read live as an upper bound on what runs and --not-live as a hard floor on what does not. Two things the axis structurally cannot see make a live verdict weaker than it looks: runtime gates (a function called every frame behind a flag never set) and partial ports (reachability is a property of the entry symbol, not its body).
--live-audit writes live-audit.md, comparing the reachability verdict against the NOT WIRED: disclosures written in the source: tagged NOT WIRED but analysed live (tag and analysis disagree - needs a human), undisclosed inert ports (a wiring gap or a missing disclosure - the reason this mode exists), and disclosed inert ports (the declared wiring worklist, working as intended).
Anchor resolution: which symbol a tag binds to
| Tag form | Anchor | Live when |
|---|---|---|
/// / // above a fn | that function | the function is reachable |
above a struct / enum / impl | that type | any method in the type's impl blocks is reachable, or - when the file gives the type no impl at all - any non-test fn in the file is |
above a const / static / type alias / macro_rules! | that item | a reachable non-test fn body references the item's name |
// inside a function body | the enclosing function | that function is reachable |
//! PORT: (module doc) | the file, widened to its submodule subtree when the file declares no functions of its own | any non-test function in scope is reachable |
A /// doc block resolves to the item it documents. Module-level tags are the coarse case and the main source of over-reporting: a //! block on a crate root claims the whole crate. Disclosure resolves per anchor: the tag's own block saying NOT WIRED discloses it; failing that, a //! NOT WIRED module doc discloses every anchor in the file; an anchor whose own doc block opens a line with WIRED: opts out of the blanket. --selftest runs the anchor-kind and disclosure-precedence cases on a synthetic corpus.
The audit compares per anchor, not per address: a formula ported into both engine-vm and engine-core has two anchors and can legitimately be wired in one and not the other. The stale-NOT WIRED test reads off a second, receiver-gated graph so a name collision does not fire on unambiguous names, while live / --not-live stay on the permissive graph - sharpening the shared one would trade the hard floor away.
History: analysis defects the audit itself turned up
Two dispatch shapes the graph could not model each hid a whole tree of live code: trait default methods (counted as methods of their trait) and winit's ApplicationHandler callbacks (a root family). A #[cfg(test)] helper is still a caller node: a test helper with a common name (step, tables, new) collects every same-named edge and marks its own module live, which reads as a stale tag on a module nothing calls - give test helpers distinctive names. Per-row verdicts live in live-audit-triage and stale-not-wired-triage.
The ratchet
The catalog's figures reach this site's landing page through scripts/ci/progress-metrics.json, so a run that could silently move a published number needed a failure mode. --check compares against the committed baseline and is a hard pre-commit gate (CI reports SKIPPED there, because the dumped column reads the gitignored Ghidra corpus - the hook is the only place these figures are compared).
| Figure | Direction |
|---|---|
worklist/port - dumped + documented, not ported, not ignored | may not grow |
worklist/dump - cited but not dumped | may not grow |
worklist/ported_not_documented | may not grow |
worklist/ported_not_dumped | may not grow |
live/disclosure_gap - inert anchors carrying no NOT WIRED: tag | may not grow |
totals/ported | may not shrink |
Everything ratcheted is a property of the tags, the docs and the dump corpus; nothing read off the receiver-gated graph is baselined, because its numbers move with graph resolution rather than with the work. Three rules protect figures that need --live: --check fails on an uncompared figure unless the caller says --allow-uncompared; the hook spends the slow pass when the commit stages crates/; and --update-baseline carries forward figures its run did not compute rather than dropping them. Validate a surprise against rows, never against the count - a regression report is a prompt to open --missing-ports and --live-audit.
History: why "every headline number was an instrument artifact"
More than one wave found the count moving for a reason inside the measurement rather than inside the tree. The disclosure gap was baselined at zero while the hook ran --check without --live; the run printed NOT COMPARED THIS RUN, exited 0, and the figure drifted with every gate green - printing a line is not a comparison. A --update-baseline run without --live once deleted the whole live block in one command. And sharpening the shared call graph to fix name collisions was tried and reverted more than once, because it traded the hard not-live floor for a fix to the opposite error - which is why the tool builds two graphs. The per-address companions: port-provenance (nothing checks that a // PORT: address names the routine the Rust code implements - a live port once wore the casino prize list's address with every gate green) and worklist-classification (whether a --missing-ports row is a portable function entry at all).
Ignore list
scripts/ci/port-catalog-ignore.toml lists addresses out-of-scope for engine porting - statically-linked PsyQ kernel / runtime / SDK code the from-scratch port maps to native equivalents (Rust stdlib, wgpu, cpal). Categories are organisational (one TOML table per cluster: bios, libgte, libsnd, ...); each entry is an address plus a one-line factual reason naming the PsyQ function.
[libgte]
"8005ba1c" = "GTE sqrt / normalise (mtc2 0xF000 / mfc2 0xF800)"
[libsnd]
"80062340" = "SsSeqOpen (slot-bitmap walk + load)"
- One section is not PsyQ infrastructure:
unreferencedholds retail-unreachable entry points - real routines nothing on the disc reaches in any reference form, per the address-reference scan. A row moves there on its subject matter, never on its reference count alone. - The port worklist is not zero and should not be: disc coverage keeps surfacing routines no citation-denominated worklist could list, and documenting one raises the worklist. That is the two measurements composing correctly - never hold the number down by declining to document a function, and never park one in the ignore list to hide it.
Settling a row whose blocker is "no caller"
Two of the ways this engine reaches code are invisible to a call-graph sweep - a function-pointer table or actor-template word, and a lui+addiu address pair Ghidra's reference manager does not resolve - so "no jal targets it" is not yet a finding, and "no SCUS caller" is never "dead". Run the address through the address-reference scan first: a table or template word means the row is real work; nothing anywhere means the address is linked but unreached (document the negative and file it under unreferenced); branch sites but no call sites means an intra-function label, not an entry.
The runtime denominator
Everything above is a static graph answering could this be reached. It cannot answer was it reached. scripts/ci/replay-port-coverage.py supplies the missing denominator by joining cargo llvm-cov output for the pad-only replay ladders against the catalog's address → (file, line) anchors.
| Set | Meaning |
|---|---|
| inert-entered | The static graph says no host reaches it; the run executed it anyway. The graph is wrong or the tag is on the wrong symbol - each row is a finding. |
| disclosed-entered | A NOT WIRED: anchor that a passing oracle executed. Highest priority: an oracle traversing stub code can certify behaviour nothing implements. The only gateable set (--fail-on-disclosed). |
| live-unentered | Statically reachable, never reached. Not a defect - the wiring worklist ordered by what a playthrough needs. |
| not observable (const) | Item anchors with no executed attributable reference. |
Running it: the ladder union, and why it stays manual
cargo llvm-cov clean --workspace
scripts/ci/replay-port-coverage.py --list-ladders | while read -r t pkg; do
cargo llvm-cov clean --profraw-only
cargo llvm-cov -p "$pkg" --test "$t" --no-report
cargo llvm-cov report --json --output-path "target/cov-$t.json"
done
scripts/ci/replay-port-coverage.py
The denominator is a union of ladders, not one binary - the world spine, the pause menu + save UI, the minigame doors, the cold-boot field anchor, the browser draw-composition ladder and the native-window ladder each reach content the others do not; canonical membership is CANONICAL_LADDERS in the script. A bare invocation globs target/cov-*.json and names any member whose export is absent. No --release: an optimised build inlines small functions and leaves their coverage record at zero, indistinguishable from never-called. The headless BootSession ladders construct no draw list, so the browser composition ladder (web-viewer/tests/play_compose_ladder.rs) and the spawned-play-window ladder (engine-shell/tests/w5_native_minigame_ladder.rs) are what execute the draw-list builders. Each export is an instrumented build plus a full disc-gated run, so this stays a manual step. Per-address verdicts for the live-but-never-entered set are in reach-triage.
Caveats
- The citation graph is dump-local. An undumped helper has no outgoing edges; the frontier widens only as dumps land.
documentedis broader than the curated functions directory. Any doc page that mentions the address counts.- One
// PORT:tag does not guarantee semantic equivalence. The tag is a provenance link, not a correctness proof; tests and retail comparison do that job. - The ignore-list is curated, not exhaustive. Newly-dumped PsyQ helpers surface in
--missing-portsuntil added; treat unfamiliar 16-byte thunks in0x8005xxxx/0x8006xxxxas likely candidates.
Companion pages (on GitHub)
The catalog has a family of per-row triage and integrity pages in docs/tooling/, not mirrored on this site.
| Page | Question it answers |
|---|---|
| worklist-classification | Is a --missing-ports row a portable function entry at all (REAL / INTERIOR / SHARED_TAIL / DUPLICATE / VA_ALIASED / ...)? |
| live-audit-triage | Per-anchor verdicts for the audit's undisclosed-inert rows. |
| stale-not-wired-triage | Per-row verdicts for "tagged NOT WIRED but analysed live". |
| reach-triage | Verdicts for the live-but-never-entered set. |
| port-provenance | Does a tag name the right routine? (Ungated.) |
| call-target-integrity | Why a decoded jal target belongs to the bytes, not the load base. |
| dump-corpus-integrity | A dump's printed addresses are a property of its load base; only the header tag is evidence. |
| phantom-print-index | For each printed 0x801C**** / 0x801D**** address, the image and VA its bytes really occupy. |