Frame benchmark: what the engine costs a player

Two drivers measure the engine on a fixed set of scenes and on the way into them: scripts/ci/frame-bench.py on the native window and scripts/ci/frame-bench-web.mjs on the browser play page. They print tables and assert nothing. A frame-time gate on a shared machine can only flake, so the numbers belong in a before / after comparison on one machine, and not in the repo. What does gate is the cost ratchet, on counts rather than times; the proof that a browser change drew the same picture is the frame hash.

At a glance

Both drivers walk the same frames on every run and vary only in the time those frames take, so two runs on one machine compare directly.

Native

scripts/ci/frame-bench.py                                  # every scene, both fidelities
scripts/ci/frame-bench.py --scene town01 --mode enhanced --repeat 5
scripts/ci/frame-bench.py --json before.json               # raw rows for a diff
scripts/ci/frame-bench.py --exe new --base-exe old --repeat 5   # the two builds alternating

The driver builds legaia-engine with engine-shell’s bench-alloc feature and runs play-window tick-locked: one sim tick per drawn frame whatever the wall clock did, so every run walks the same frames and only the time varies. The audio director runs over a device-less output. It prints the fastest of --repeat runs with the spread between them, which says how far to trust a row on a busy box.

Table	Reads
Frame time	average, p99, worst, allocations and bytes per frame, draw counts, the process’s peak RSS, then time per stage
Allocations by stage	which stage the allocations happen in
Startup by step	process up to first frame: renderer and pipelines, VRAM, each mesh family, actors

The stages are the frame profiler’s. The worst frame’s own stage breakdown goes to stderr beside each row, which is how a hitch gets a name. LEGAIA_PROFILE_ALLOC_SITES=1 on a bench-alloc binary samples allocation call stacks and prints the heaviest at the end, as [alloc-site] lines.

Entry

scripts/ci/frame-bench.py --entry --base-exe old --repeat 7   # paired against another build
scripts/ci/frame-bench.py --entry --entry-scenes town01,koin1

legaia-engine entry-bench enters each scene and forces a fight from it with no window and no GPU, one [entry] <scene> <step> <ms> line per step. The first round enters through the scene picker and carries the once-per-process caches; the second, labelled scene+, is what a door back into the scene costs. What a window adds past these steps (surface, pipelines, mesh upload, draw lists) is the frame table’s startup row.

Browser

scripts/ci/build-wasm.sh                                      # or point --site at a scratch copy
scripts/ci/frame-bench-web.mjs --scene town01,battle --profile
scripts/ci/frame-bench-web.mjs --site /path/to/other/site --json out.json
scripts/ci/frame-bench-web.mjs --paths --vsync                # the player's ways in; at the display's rate
scripts/ci/frame-bench-web.mjs --cpu-throttle 6               # a proxy, labelled one

It serves a site/ directory itself, loads the disc named by LEGAIA_DISC_BIN through the page’s own file input, and enters each scene through the page’s own scene list. Each row reports the page’s frame function as a main-thread cost, every wasm export’s time, calls and returned bytes per frame, GL calls per frame, long tasks, and the scene-entry time.

Flag	Adds
`--profile`	A CPU profile: self time by function and by kind (JS, wasm, GL calls, the rest), garbage collector included, an inclusive table, and the call chains of the heaviest.
`--screens [scene]`	The long tasks outside scene and battle entry: the pause menu’s rows, a talk, a first shop, or with `map01` a whole fight to its results. Each step’s worst frame, then the longest frames as call trees.
`--trace`	A browser trace over six seconds: main-thread self time per trace event, inside and outside the animation-frame callback. It names what a profile calls `(program)`: style, layout, paint. `--trace-out file` writes the events, invalidation tracking included.
`--gl egl` / `swiftshader`	The default asks for a real GPU and the output names the renderer it got; a software rasteriser swamps the page’s cost with fragment work.
`--entry-profile`	Samples the disc load, each scene entry and each battle’s first frames; time by function, self and inclusive. A battle row adds its three longest frames as call trees; read those with `--vsync`.
`--reenter`	Enters each scene twice in a row.
`--reload`	Reloads the page once the disc is up and times a returning visitor’s load from the page’s own disc cache.
`--idle ms`	Waits at the boot-mode prompt before the first pick, which is when the page prewarms the first entry.
`--vsync`	Leaves the browser’s frame limit on, so the cadence row reads whether the page holds the display’s rate rather than how fast it can go. It moves the frame function too: uncapped, the main thread blocks on the GPU inside its GL calls, so compare like with like.
`--paths`	A New Game and a card load, each on a fresh page; `--card` and `--card-block` pick the save.
`--cpu-throttle n`	Slows the renderer’s main thread. A proxy, and the output says so: it shows which costs scale with the CPU and says nothing of a device. Read it with `--vsync`: uncapped, the page queues GPU work until a backpressure block of over a second, which no capped page builds.
`--uncheck id,id`, `--no-msaa`, `--canvas-scale s`	Clear option boxes, ask for a context without MSAA, draw the GL canvas at a fraction of its size. With `--gl swiftshader --vsync` they read what one rung of quality costs on a GPU short of it; a tiler may rank them differently.

The summary line reports the head of the disc’s read, the disc-load export (load_disc_spans) and the module’s memory size. A play bundle built with web-play-bundle’s bench-alloc feature adds wasm allocations per frame. A wasm profile names its functions by index; wasm-bindgen --keep-debug over the cargo artifact gives a file whose name section resolves them.

What the frame function leaves out

Each browser scene row carries these beside it, so one run answers where a weak device pays.

Row	Reads	Read it as
`gpu span`	The timer query around the frame function	An upper bound on GPU work: the query spans the wait for the main thread’s next commands, so its lower edge is the frame function
`cadence`	Gaps between frame-function starts, and how many ran over 33 ms	What the display got, with `--vsync`
`main thread per frame`	The renderer’s own task time, less the frame function	What runs outside it. The audio pump is named: the worklet only plays blocks, and the mix runs in its report handler on the main thread
`audio across the battle edge`	`audio_stats_json` before the force and after the hand-over	Dry frames a stall cost. A run ends with a 1000 ms stall that has to read dry frames, or the output says the row proves nothing
`memory MB`	Process RSS off `/proc`, wasm, resident disc bytes, JS heap, a GL tally	Who holds what. The tally is what the page asked for, not the driver’s account

A battle row waits for the page’s battle scene, not a frame count: the intro runs on sim ticks, so on a page drawing faster than the sim ticks a frame count ends inside it. The second_fight scene plays a first fight to its end, results included, then forces the fight the row times, which is where a memo would show.

Expansion packs

scripts/ci/pack-bench-web.mjs --maps genesis_1f,isles_beach        # the workbench's painter
scripts/ci/pack-bench-web.mjs --entry --map isles_beach             # the play page's first map
scripts/ci/pack-bench-web.mjs --cpu-throttle 4 --profile            # a proxy, labelled one

The painter's rows run from a single painted tile: what the press and its release hold the page, the frame that shows the tile, when the map_preview left, when its answer was on the canvas, and the worker's share. A CPU throttle reaches the page's thread and not the worker. --entry reports the library row's Enable up to the scene list, then the click on a map up to its second frame, with the top wasm exports of each. Native startup on a pack map is frame-bench.py's pack_map row.

Another machine

python scripts/ci/machine-bench.py --disc game.bin --extracted extracted --exe new/legaia-engine.exe --base-exe old/legaia-engine.exe --site new-site --base-site old-site

scripts/ci/machine-bench.py runs one machine's base against tip with no Rust or wasm toolchain: two release binaries and two built sites are its inputs. It runs the native frame and entry tables, then the page's rows at a display's pace, uncapped and under CPU throttle, and zips the logs, JSON and a description of the machine for sending back. The archive holds times and counts, with no screenshots and no disc bytes.

It runs on Linux and Windows. On Windows the page's --gl egl means ANGLE's Direct3D backend, the process memory rows read empty, and a native row's peak RSS reads 0. A release binary has no bench-alloc, so its allocations read 0 too.

Movie start

scripts/ci/movie-timeline-web.mjs times a movie's start on the play page, hooked from outside so a player's page carries none of it. Each run prints a JSON line, from the request: span resident, install, first decode, first picture, first sound (the mixer's cursor passing zero, ahead of the device by its queue), first motion, the worst display-frame gap, long tasks, wasm memory and the box's busy share over the window.

scripts/ci/movie-timeline-web.mjs --again --append out.jsonl                 # a read, then a second play
scripts/ci/movie-timeline-web.mjs --evict --again --append out.jsonl         # the read reaches storage
scripts/ci/movie-timeline-web.mjs --route attract --append out.jsonl         # the title's countdown arms MV1
scripts/ci/movie-timeline-web.mjs --read-delay 9000 --watch 13000            # walk the install timeout

It is one start per run, because the claim it tests is intermittent: the distribution is the caller's, over a run count, with each run under the display lock. --evict drops the disc from the OS page cache. --ctx-suspend stands in for an output slow to start, since the driver's browser has no autoplay gate. --read-delay holds only the movie's own read, in flight; --no-fill and --no-fill-wait matter only to an older bundle's fill (--site). --cpu-throttle and --gl swiftshader are proxies.

Cost ratchet

scripts/ci/cost-ratchet.py --check                # the gate, against scripts/ci/cost-baseline.json
scripts/ci/cost-ratchet.py --measure --runs 10    # the spread between runs, nothing compared
scripts/ci/cost-ratchet.py --write-baseline       # after a real change, with its ledger entry

Times flake and counts should not, so this ratchets counts, each from a tick-locked run that walks the same frames every time. It is the cost-ratchet gate in full-gate.py. A rise past a count’s room fails; raising the baseline is a loosening, which check-baseline-loosening.py sends to the ledger with its reason.

Count	Read from	Spread over ten runs of one tree	Room
Bundle gzip bytes	The shipped wasm	None	0.5%
Browser GL draws per frame	`frame-hash-web.mjs --counts`	None	None
Browser wasm bytes returned per frame	The same	None	0.5%
Native draws per frame	`play-window`’s `[bench]` line	None	None
Native allocations per frame	The same, on a `bench-alloc` binary	Under 0.05%	0.5%
Native KB allocated per frame	The same	Under 0.05%	0.5%

None of the room covers a wobble. It is the size of rise not worth a ledger entry; a draw always gets one. The bundle is read only off the shipped build: a --fast bundle is a [skip], and a skip is not a pass.

What the numbers have shown

These are shapes that recur, not measurements.

A posed body re-parses its TMD every frame. Both hosts’ posed builders walk every primitive, which is why a primitive’s corner streams are held inline rather than as four heap lists each.
A per-tick copy can live in one host only. An owned read of a whole PROT entry in a browser export’s tick path is invisible on the native window, which holds its own copy. The per-export table is what shows the difference.
Shadow layers multiply the draw count. A draw wholly outside a lamp’s frustum writes no depth to that layer, so both hosts test each mesh’s box per light.
A first entry pays for what a scene never reads. The 64 cast modules parse on a module’s first read, the fog ground on the first tick with the toggle up, and a scene load asks for an entry’s class without the byte statistics a report carries. Each deferred value is a pure function of what the load captured, so its reader gets what it would have had.
Whole-disc copies cost at the wasm boundary. A wasm memory never shrinks. The page’s image stays on its side of the boundary and load_disc_spans copies across only the files the runtime keeps.
A page need not read what it may never play. The image is the largest read on the way in, and two thirds of it is movies and XA, which reach the page behind the engine (the disc image).
The scene picker’s first pick walks every scene. The arrival catalog reads each scene’s MAN and .MAP to find its doors, once per process. A door, a New Game and a card load do not build it.
The capture frame’s cost is the GPU, not the bytes. readPixels on the battle transition’s arming frame waits for that frame to finish, and the wait does not shrink with the size of the read. An asynchronous read would land the capture frames later, which moves the transition.
The worst battle-edge frame is the hand-over, not the capture. Where the transition hands over, one frame runs the fight’s first tick and builds and uploads its whole render. Decoding the party’s battle files and building the stage were the largest items; a memo holds them across fights and the transition’s frames fill it ahead of a first fight.
A first fight’s hand-over pays for cold code. The same frame in a second fight is a fraction of the first’s, and no memo reaches that.
An uncapped row’s entry can be the GPU’s queue. With the frame limit off, the first bufferData after a heavy row blocks until the queue drains. With --vsync it is gone.