Build and test speed

Which profile

dev-fast binaries lose some inlining across codegen units and run a few percent slower for it on the ladder, with the whole suite’s outcomes the same, test for test. CI stays on release-test, and a change is not verified until it has run there. dev-fast is a local choice and no script or workflow names it.

What an edit costs

A crate is compiled whole when it or anything under it changes, because cargo has no early cut-off on an unchanged interface. So the cost of an edit is the crates above it. An edit in engine-core recompiles everything from engine-scene-host up to engine-shell’s integration test binary, each from scratch under release-test. dev-fast recompiles the same crates, and each reuses the codegen units the edit did not reach; that is the whole difference, and it is an order of magnitude on a mid-graph edit.

• LLVM dominates. Type-checking and borrow-checking are a small part of nearly every crate.
• legaia-expansion-format costs the most of anything in the graph, for its derived TOML types, but it is low in the graph and changes rarely, so it weighs on a clean build and not on an edit.
• engine-shell’s integration binary is the largest single unit an edit reaches. An edit under crates/engine-shell/tests/ rebuilds that unit alone.
• sccache misses on workspace crates between checkouts by design, since the checkout path is in the crate’s metadata. What it serves is dependencies.

The wasm bundle

The shipped bundles are scripts/ci/build-wasm.sh: release plus wasm-opt stripping, the tool bundle into site/wasm/ and the play bundle into site/wasm/play/. They share every rlib, so the second costs one more link and strip. A one-line edit in engine-core costs three quarters of a clean build, and no one step is why: it is the chain above the edit, each crate compiled whole, then the link. So the iteration path is a separate build.

wasm-fast lands in site/wasm/ where a page expects it, and the freshness stamp records the profile: check-wasm-freshness.py --strict refuses it, so neither a claim that a fix is live nor the full gate can rest on it. scripts/ci/check-wasm-target.sh stays the cheapest question, whether the browser host compiles at all.

The linker

The aarch64-unknown-linux-gnu toolchain links with the system cc, which runs GNU ld. rust-lld, which rustup ships, takes well under half the time on the test binaries. Set it per shell and not in ~/.cargo/config.toml: on a machine that is also the CI runner, that file applies to the release builds as well, and release-build.sh is the only thing that should decide how a shipped binary links.

Where a test run goes

The suite’s wall clock is set by the slowest binary, since cargo runs binaries one at a time and tests within one in parallel. With a disc, that is engine-shell’s integration, whose length is part C of the full-game ladder. patcher’s integration is the most CPU, nearly all of it in the tests that apply a whole mod to a whole disc. libtest’s per-test timing works on a stable compiler when RUSTC_BOOTSTRAP=1 and -Zunstable-options --report-time are passed to the built binary, run from the crate directory.

Page faults. A test that holds the disc image in a Vec pays twice: the copy, and a page fault for every 4 KiB of the allocation. On the patcher suite those faults are more CPU than the tests’ own code. The repo’s .cargo/config.toml sets GLIBC_TUNABLES=glibc.malloc.hugetlb=1 for everything cargo spawns, so glibc asks for transparent huge pages behind its large allocations and one fault covers 2 MiB. It moves nothing a program computes, and it reaches only what cargo runs.

The commit-time gates

The pre-commit hook runs the same checks in the same tree whatever the order, so the order is chosen for how fast a failure reports.

1. The sub-second gates, one after another.
2. cargo fmt, with its auto-fix, when Rust is staged: whatever reads source below reads what the commit will hold.
3. The gates that take seconds, side by side: the dump-corpus ratchets, the port catalog, and on a Rust commit the host-drift gates. Every failure is reported, not only the first.
4. cargo clippy and the wasm32 check, which are the longest and take cargo’s build lock.

A gate that is slow is usually repeating itself. For a gate, the proof of a speedup is its output: port-catalog.py --live-audit must write target/port-catalog/ byte for byte as before.

The full gate

scripts/ci/full-gate.py is the end-of-work gate as one command. It reports and never decides: nothing it runs blesses, proposes or pushes, and a command that carries such a flag is refused before the run starts.

The workflow’s steps are read from the file on every run, so a step added there is a gate here with no edit. Every failure is reported with its log: a gate whose prerequisite failed shows as BLOCKED, and the rest still run. A driver’s [skip] line reads as SKIP and the verdict as PARTIAL, since a skip is not a pass; only the repo group is exempt, where a ratchet skipping for want of the disc is what the runner does. preflight fails when the data a gate needs is missing, and tree-unchanged runs last and fails if a tracked file moved during the run.

Output goes under --out (default target/full-gate): one log per gate, timing.md (each gate’s start, length and CPU seconds, which are what a critical path and a gate’s real share of the budget are read from) and state.json. --max-cpu, --jobs, --cargo-jobs and --memory-max set the load on a shared box.

At the default budget of twelve and six jobs, two heavy gates run side by side. What sets the wall clock is the chain the display lock serialises: retail-compare, the two smokes, pages, quality-ladder, cost-ratchet. The scheduler offers that chain and the cargo-rt chain under the tests the budget first, with the lints among them, and the seeds last: they block nothing and each takes one core.

Profiling without perf

Where perf is refused, scripts/profile/sampler.c gets the same answer with no privilege and no rebuild: a preload library that takes a stack every 10 ms of CPU time, and scripts/profile/report.py to read it. On a shared machine compare CPU seconds, not wall clock, and compare two binaries by running them side by side.

A speedup is proved by the output, not the timing. For part C of the ladder, that is the trace with its wall-clock fields stripped, which repeats on one binary, so a difference is the change's; for a patcher feature, the .ppf from a fixed seed must hash the same.

See also