Soak harness: random play that hunts softlocks
The route ladders walk a path someone chose and score how far it gets. The soak harness does the opposite: it drives the headless engine with seeded, game-shaped random pad input across every playable scene and every minigame, and reports whatever breaks - a confirm pressed during a door fade, the pause menu opened on the frame a battle starts, a direction held into a picker. Every finding is saved as a j-replay-v1 pad script that reproduces it deterministically, reduced to the input that matters.
At a glance
- Test
crates/engine-shell/tests/soak_harness.rs- disc-gated, skips and passes withoutLEGAIA_DISC_BIN- One run
(scene, seed, frames)- nothing else; the input is a pure function of the seed and the observed session- Actuator
World::set_pad, once per frame- Scene set
- every playable CDNAME scene, the
0x3Fdestinations of those scenes, the five minigames, plus+rt,+talkand+shoptwins - nothing hand-listed - Output
target/soak/<tag>/:report.md,findings.json, one reduced replay per signature- Regressions
scripts/replays/soak/fixed/- closed findings that must never fire again
(scene, seed, frames) regenerates a run exactly. A finding leaves the loop as a reduced replay; once a fix stops it reproducing, it becomes a regression.The driver
Each run opens a fresh BootSession, seeds the retail New Game party, mounts an empty two-port card rack, seeds the world RNG from the run seed and enters the scene live with the random encounter loop and player-driven battles armed. Then it ticks.
BootSession::tick is not a whole host. Both play hosts do several things around it every frame, and a driver that skipped them would measure its own omissions. The harness does what the hosts do:
| Host duty | What the harness does |
|---|---|
| faithful-play arming | what both hosts set on a bare session: inline field-VM talk, the leading-edge wall footprint, solid NPCs, terrain follow, the NPC ambient walk |
| presentation queues | drains the field-event, battle-event, hit, SFX, shout, XA, CLUT-stage and minigame-cue queues after every tick |
| FMV playback | skips the movie and runs the shared post-play hand-off |
| name entry | routes the pad edge into name entry and skips the field tick while the overlay is up |
| shop / prize counter | opens a staged shop in a MenuRuntime, steps it on the frame's edges, and unparks the field script when it closes |
| save screen | a Save writes into an in-memory card; a Load resumes that block through resume_save |
Pseudo-scenes
<venue>+mg<id>- A minigame: the venue is entered, then the same mode-24 door warp the cabinet's op
0x3Emakes is armed. The party is given casino coins first - the one non-pad precondition. <scene>+rt- A save / load round trip on every free-roam frame a player could save on, at a fixed interval: save, resume that file as a Load does, save again - the two must agree field for field. The run starts with one save on the card, so the pause menu's Load row has a file to resume.
<scene>+shop- Every scene with a priced gold shop: the scene's shops are handed to the menu runtime as the merchant's op
0x49would, since random walking almost never reaches a merchant and picks Buy. <scene>+talk- Every field scene: the player is stood in front of the next action-button target - NPCs first, then interact-gated props that carry a record - facing it, and the button is pressed. Encounters are disarmed; the run is there for the records.
<label>+rage- Every roster member carries the Evil Medallion's Rage passive for the whole run, so the anim tick delegates the whole party in every battle: the command ring refuses every member, and each round must still open on
Begin | Run, reach the commit confirm and play out on the delegated picks. ...@<save>- Any label can lift a library-card save (
town01+rt@PRO-14) and play the scene with that save's party, bag and story flags, so late-game scenes are soaked with the party they are reached with.
A talk seat is taken only where play could put the player: on authored walk-visible floor in an open component, somewhere the walk controller's own probes let the player walk a tile away and come back. An actor is faced from 112 units where it can be (a face-on walk stops 104 short of a standing NPC), a prop from 128. The seat is a function of the frame and the world, so a replay takes the same seats.
Round trips caught a real defect this way: a field save folded actor slots 1 and 2 back into the party records, but in a field those slots belong to the scene, so opurud+rt lost two party records on resume. World::save_party now folds actor mirrors back only in a battle. The library-wide sibling, engine-session/tests/save_roundtrip_library.rs, round-trips every save on the library cards through the card path both hosts take.
Policies
What the policy presses depends on what holds the frame. Every UI surface reads just_pressed, so a press is a tap: the wanted mask on one frame, neutral on the next.
| State | Policy |
|---|---|
| field / overworld | segments of held walks (straight or diagonal, sometimes with Cross taps), Cross / Circle mashing, a Start tap, idle, and chaos (random masks of everything but Start) |
| dialogue, or a script holding the player | mostly Cross, some directions, some Circle |
| a script polling the held pad | the face button it polls, held - op 0x42 mode 1 reads the held word on one frame of its loop, which a tap hits only by luck |
| a picker | a random number of Down taps, then Cross, once the box's slide has come to rest; the aimed row steps on each time the same picker returns, so a quiz that asks again gets every answer |
| pause menu | random taps for a random budget, then Circle until it closes |
| battle | mostly the route ladder's fighter (Begin, Attack, Auto, confirm); in the sparring tutorial it follows the lesson, since any other pick rewinds it |
| minigame / other | random face buttons and held directions, rarely Start - the minigames' escape |
Detectors
| Detector | Fires when |
|---|---|
panic / tick_error | a tick, scene entry or FMV hand-off panics or returns an error |
hang | one tick runs past the wall-time watchdog; it writes HANG.txt and exits |
softlock | the progress digest is unchanged for a whole window while the pad took at least three distinct masks |
script_stall | the modal timeline or first helper sits on one PC for a window of field frames - unless its scripted walkers are still stepping, or it is a walk-on record re-polled every tick |
battle_loop | three battles open from one parked timeline PC |
menu_stuck / battle_endless | the pause menu survives the Circle back-out; one battle outlasts its threshold |
control_starved | the field refuses the pad with no free-roam frame, the script reaches no new PC, and confirm presses are still arriving |
confined | 900 direction-held frames, all four directions among them, never carry the player 48 units |
door_loop | six scene changes inside 600 frames with no direction pressed between them |
dropped_to_title / unknown_scene | the mode becomes Title without a game over; a transition names no scene in the set |
non_finite | a camera float or the caption alpha is NaN or infinite |
value | HP / MP over max, live maxima that are not the stat aggregator's rebuild, a level outside 1..=99, a bag stack over 99, a count on a free slot, a duplicated stack |
unbounded_growth | an engine queue grows past a cap |
effect_residue | a battle effect is still live on the first field frame after a battle exit or scene load - retail resets the whole pool there (effect VM) |
timeline_cap | the engine's anti-hang frame cap ended a script context; retail has no such cap, so the record was cut short |
save_roundtrip | in a +rt run, the post-resume save disagrees; the location names the first differing field |
The progress digest hashes everything a player could see move: mode, scene, position and heading, every script PC, dialogue, menu, shop and name-entry state, scripted NPC glides, the battle session and every actor's HP, the minigame session, money and bag size. It is deliberately broad: a false "frozen" costs a triage, a false "moving" only a missed finding.
A softlock is located by what holds the frame - pause-menu:<row>:<phase>, battle:state:<action state>, timeline@pc:op, mode:<minigame>, free-roam:immobile@tile(x,z) and so on. A battle park on an absorbing HP-bar pair (the shape that holds the bar-drain gate for good) carries a :hp-bar-absorbing suffix.
control_starvedcatches a script going round. The softlock and stall clocks restart whenever anything moves, so a conversation that reopens its own question passes both forever. Its clock restarts only on free roam, a scene change, or a PC this stretch has not seen.confinedis measured, not predicted. It uses the walk controller: the leading-edge probes let a player rest where a flood of the grid says walled, and an actor can shut a seat the grid calls open.door_loopignores a held direction. A two-way stair sets the player one tile from the door back, so a held direction walks straight back in.
Findings, signatures and reports
A finding's signature is detector|scene|location. Findings are deduplicated by signature and ranked: panics, hangs and tick errors first; softlocks, loops, stuck menus, endless battles, title drops and unknown scenes next; immobility, stalls, starvation and confinement after; value, non-finite, growth and round-trip checks last.
- Confirmthe first occurrence is replayed from its pads, truncated to the finding frame, and must fire again
- Reducechunks of the pad stream are zeroed while the signature still fires - a bounded delta-debugging pass
- Write
replays/<slug>.replay.toml, with scene, seed and signature in header comments TOML ignores
A softlock is reduced under the neutral-pad control, so input that only fed the frozen window drops out; the replay records that in a # soak-min-distinct header. A control_starved finding is reduced only ahead of its window, because a stream cut down to one looping option would starve a healthy conversation too. Every replay names the session configuration it ran under (# soak-arming = "host" or "bare", with a -retail suffix under the hosts' retail mode).
Running
The fixed-budget gate runs a few scenes plus the five minigames and asserts no panic:
cargo test -p legaia-engine-shell --profile release-test --test integration soak_harness::soak_smokeA long soak is report-only unless LEGAIA_SOAK_STRICT=1:
LEGAIA_SOAK_SEEDS=30 LEGAIA_SOAK_FRAMES=18000 LEGAIA_SOAK_JOBS=8 LEGAIA_SOAK_TAG=big cargo test -p legaia-engine-shell --profile release-test --test integration soak_harness::soak_long -- --nocapture| Variable | Meaning |
|---|---|
LEGAIA_DISC_BIN | required; without it every disc-gated test skips |
LEGAIA_SOAK_SEEDS / _SEED_BASE / _FRAMES | seed count, first seed, frames per run |
LEGAIA_SOAK_SCENES / _SHARD | an explicit scene list; i/n to chunk a long soak across invocations |
LEGAIA_SOAK_JOBS / _TAG / _OUT | worker threads; report directory name and base |
LEGAIA_SOAK_SOFTLOCK_FRAMES / _STARVE_FRAMES / _BATTLE_FRAMES / _HANG_SECS | detector windows and the single-tick watchdog |
LEGAIA_SOAK_MINIMIZE / _CONFIRM_MAX / _NO_CONFIRM | reduction budget per signature; how many to confirm; skip confirmation |
LEGAIA_SOAK_SAVE / _CARD | play every run from a library-card save |
LEGAIA_SOAK_NO_ENCOUNTERS | disarm the encounter roll - a battle that still starts is scripted |
LEGAIA_SOAK_RETAIL | soak under the hosts' retail mode: the options' engine knobs and every fidelity toggle pinned to retail |
LEGAIA_SOAK_BARE | soak a bare BootSession without the hosts' arming |
LEGAIA_SOAK_REDUCE / _REDUCE_SIG / _REDUCE_OUT | re-reduce a saved replay, optionally toward a different signature |
The headless engine ticks far faster than real time, so a soak is bounded by the minimiser, not by the runs. Overflow and debug-assertion coverage comes from building the same test with those checks on, into its own target directory.
Reproducing and triaging a finding
LEGAIA_SOAK_REPLAY=target/soak/big/replays/<slug>.replay.toml LEGAIA_SOAK_TRACE=60 cargo test -p legaia-engine-shell --profile release-test --test integration soak_harness::soak_replay -- --nocaptureLEGAIA_SOAK_TRACE=N prints a state line every N frames - mode, scene, holder, position, the bytes at every live script PC, scripted glides, and in battle the action state and every actor's HP / display / accumulator - plus each distinct script body once, as hex. LEGAIA_SOAK_TRACE_NPC=<slot> follows one placement; LEGAIA_SOAK_TRACE_WALLS=1 draws the wall bits around the player. The trace goes to the terminal only.
A fix is checked both ways: the replay must stop firing on the fixed engine, and must still fire on the engine without the fix under the harness as it now stands - a detector that changed since the replay was written can leave it passing for the wrong reason.
Fixtures
scripts/replays/soak/- Reduced replays of open findings, pad input only.
soak_fixturesparses each without a disc and, with one, reports whether it still reproduces. It never fails on one that stopped: that is the signal a fix landed, and the fixture moves tofixed/. scripts/replays/soak/fixed/- Closed findings as regressions.
soak_fixed_fixturesfails if a recorded signature fires again; a# soak-expect-max-battlesheader also bounds the battles a replay may open. Only the recorded signature is asserted.
What the fixed replays pinned
| Fixed replay | Finding | What closed it |
|---|---|---|
town0c_scripted_battle_loop | the Rim Elm bee fight re-fired on every return | the touch-resumed beat ends at its 0x21; the system script sits out a held player and the battle intro |
town0d_dual_player_walk | two first-talk records walked the player at once | a playing timeline steps only the placements it addresses |
rugi_ / urudre3_npc_walk_from_hide_box | a placed NPC walked in from the off-map hide box | a seat poke that moves an actor is published at once |
bylon_party_actor_from_hide_box | a party actor walked in from the hide box | 4C 37 copies the player onto the actor |
urudre1_player_seat_dropped | a scripted player walk ran 8000 units | a record's player seat takes the player arm |
vell_attack_short_step_park | an attack approach parked a short step from its target | +0x3C is the per-frame body pair |
taiku2_hp_bar_absorbing_park | the 0x51 bar-drain gate parked on an absorbing HP-bar pair | the restaged action clip replays |
station3_ / conc3_door_of_light_head_define | a Door of Light with an all-zero return triple warped to init_data | a travel word inside the TOC header rows is a miss |
ropeway_player_parked_on_gondola | Octam's arrival cutscene left the player seated on the gondola | only the scene-init sweep's placements collide (FUN_801CF754 walks the +0x0C list) |
edlast_ending_soft_reset | after the credits the party stood with no way out | 49 0C runs the return-to-title soft reset |
koin2_fatal_decision_stone_party | a fight sat in Fatal Decision's capture band for good | the 0x5A wipe scan reads the packed status word, so a petrified party wipes |
map01_overworld_beat_walk_never_steps | an overworld hand-off beat parked on its walk wait | the world-map arm steps cross-context walk legs |
opurud_field_save_zeroes_party_records | a round trip lost party records 1 and 2 | actor mirrors fold back only in a battle |
balden_fishing_exchange_pad_dead | the pond's prize list answered no pad input | the list steps off the pad edge |
bubu2_bag_stack_past_99 | a battle grant raised a stack of 99 to 100 | every bag grant goes through the retail add, capped at 99 |
jouina_final_heal_readout_overshoot | a Final-Healed member parked the bar-drain gate | the Final Heal sweep re-syncs the readout first |
retockin_monster_curse_folded_as_capture | a monster's Curse took a party member off the field | a capture roll downs only a monster seat |
town0d_tetsu_talk_never_ends / izumi_spring_question_never_ends | a conversation or prop restarted on every way out | a touch-resumed context ends at the parking byte after a dismissed box |
map01_life_water_live_max_stale / jou_level_up_bakes_accessory_into_base | live maxima drifted from the aggregator's rebuild | stat-ups and level-ups grow the base window; the live maxima are rebuilt from it |
jouind_ledge_hop_walled_landing | a diagonal step-up hop boxed the player in a raised pocket | retail tests no landing; the port's landing guard (an enhancement, off under --retail) refuses it |
edlast_credits_end_takes_a_held_confirm | the credits parked on their last loop | not an engine defect: the loop polls a held button; the policy now holds it |
jouina_terra_summon_hits_for_one | Noa's Terra summon hit for 1 | every player summon body is its own summon.dat record |
The full list, with every fixed replay, is in the source doc.
What it cannot detect
- Visual or audio wrongness. No frame is rendered and no sample mixed; garbage on screen reads as healthy while the state advances.
- Retail divergence. Every detector is an internal-consistency check. Wrong damage, a wrong landing tile, a script branch retail does not take - all look like progress. The parity oracles own that question.
- Anything past the first fatal finding in a run, or past a game over.
- Deep progression. A plain run starts from a New Game party, which loses most late-game fights; an
@<save>run reaches only the states the library card holds. - Host drift. A duty a real host performs that the harness does not mirror is invisible here, and one the harness performs that a host has dropped is masked.