Audio

At a glance

Hardware

The SPU: 24 voices, 512 KB of its own RAM, ADSR envelopes, one global reverb. Retail runs reverb Studio C, always on, with most voices routed through it

Music

Every real track lives in the music_01 bank as a self-contained [VAB][SEQ] pair; scenes pick one by script (field-VM op 0x35, id 2000 + slot). No BGM table exists

Effects

100 static descriptors at DAT_8006F198; the descriptor's category byte names which of twelve VAB slots the cue plays from

Voices

CD-XA streams: arts shouts from XA2 / XA4 / XA6, the melee grunt from XA30, victory lines from the tail of monster.snd

Libraries

PsyQ libsnd (SsAPI sequencer, 0x80061..0x80067) and libspu (0x80068..0x8006D), statically linked and traceable

Port

crates/engine-audio: from-scratch SPU (ADSR, ADPCM, reverb network), sample-locked Sequencer, SFX bank + scheduler, XA mixing; WebAudio backend for the browser

Oracle

legaia-engine audio-trace - per-frame voice activity against save states and PCSX-Redux captures

Confidence

Confirmed for dispatch, banks, cues and the sequencer; the bse.dat record columns are the SFX-table columns, pinned by the shared decoder FUN_80016B6C

How it works

The SPU cannot be driven from main RAM directly; games go through Sony's PsyQ libraries - libsnd for sequencer playback and libspu for voice and register control. Legaia links both statically, so the libraries are part of the game binary and can be traced like any other code. Legaia's own layer above them is small: a path-string cluster, two loaders, one bank installer, one SFX helper, one BGM poller and one CD-XA clip player.

The surprising part is music dispatch: there is no "BGM id to file" table anywhere. A scene's own script writes a track id, and a per-frame poller turns that id into a PROT.DAT entry number by arithmetic over the disc layout. The archive is the table.

Finding the sound files

Retail reads sound data by PROT.DAT entry number. A dev-station path (h:\...) survives in the same functions but is never taken on a retail disc: the split is a build-mode global, and the dev arm opens files through a host trap that has no disc to read.

A third "mode-aware extension dispatcher" that turns up in call graphs is not a sound consumer at all - it is the character-mesh equipment group-transform swap (character mesh).

Sound banks and their slots

An instrument bank (VAB, format on vab) is a set of samples plus the tuning and envelope data that makes them instruments. Legaia keeps several resident at once, each parked in a numbered SPU slot. One installer opens every bank; its argument is the same category byte the SFX descriptors carry, and a 12-byte mixer record per category names the header buffer and the slot.

• Slots 2 and 6 share one SPU base. The record initialiser assigns header buffers from one base with four pairs sharing one, and the SPU-base table gives those pairs one address - the field bank and the battle class-2 bank are the same physical bank in two modes. The engine seeds the same twelve-word image at boot and answers slot queries from the other end, with a test that the two agree.
• Monster banks come from one packed file, monster.snd, with a per-monster TOC of [start_lba, end_lba + 1] pairs. Its tail clips carry the party's victory-pose voice lines: the spoils routine picks a pose tier from the actor's HP (three-quarters, half, quarter; a status forces tier 4), an action id from a per-character table, and a clip byte - Vahn 0xB8..0xBC, Gala 0xBD..0xC3, Noa 0xC4..0xCB.
• Multi-bank archives sit at extraction 0889 / 0890 (sound_data2) and 0891 (level_up). Slot sizes and the SPU map: sfx table.

Sound effects

A footstep, a menu blip, a sword clang: each is one 8-byte descriptor - which program in which bank, how many voices, how loud - and the actor or script that wants the sound just names it. The static table holds 100 descriptors; ids at or above 0x200 come from the runtime .dpk bank instead.

• Trigger. The actor tick fires the per-actor helper when the actor's one-shot pulse byte is set or a continuous cue is staged; move-VM and field-VM opcodes write the sound id and voice into the actor. The helper keys one voice per descriptor count.
• Bank is named by the cue. The voice programmer issues a VAB program-change before the program lookup, repointing libsnd's current-bank globals at the slot the category byte names. A cue never plays out of the scene's music VAB.
• Engine. SfxBank::from_descriptors builds the catalog from the disc table and fires each cue against the bank its category names; SfxScheduler dispatches at the retail timing offset (a hit cue with timing_frames = 1 plays one frame after the strike begins). Implementation crates/engine-audio/src/sfx.rs.

Music: which track plays, and how it starts

Walk into Rim Elm and the town theme starts; walk into a fight and the battle theme takes over. Nothing in the executable maps a place to a tune. The scene's own script says "play track 2016" through field-VM op 0x35, and the per-frame poller turns that id into a disc entry:

• The bank is piecewise. Global id 2000 + i equals sound-test track i, but in extraction space the bank runs 988 + i for i ≤ 67 and 990 + i for i ≥ 68, with a two-entry gap at 1056 / 1057. music_labels::prot_entry_for_bgm_id owns that map; a flat base gets the top of the sound test wrong.
• Each entry is self-contained - chunk header, pBAV body, then a pQES score. Playing a track uploads that entry's VAB into SPU RAM; a scene's scene_vab_stream SEQ is never its music, which is why fingerprinting a playing track against the scene-local corpus fails.
• Script-selected, not table-driven. Running every scene's prescript and observing the emitted id (bgm_scene_resolution.rs) shows every scene that starts music picks a global-pool id. The sweep sees a scene's entry track only; cutscenes change music mid-scene with sub-op 9 from timeline records.
• Re-selecting the playing track is a no-op - the resolved index is compared with the loaded one before anything happens.

The track-swap handshake

A cutscene can choose the exact beat the outgoing score dies on. The poller runs a stage counter through a seven-entry ladder - wait for CD idle, arm a 30-frame settle, tear down the BGM slot unless the script owns the swap, kick the load, count down, install the SEQ, re-attach unless paused, latch - coordinated with the script through one sound-flag word:

After setting bit 3 the poller stalls while bit 0 is set and bit 4 is not: a swap started by sub-op 9 keeps the old track until the script commits with sub-op 0xA. A port that honours the sub-op 2 pause but drops the 0xA commit leaves the music paused after the cutscene; the engine routes sub-op 0xA straight to BgmDirector::unhalt_pause because its BGM bytes resolve synchronously.

• Stop, pause, replay. The control words are not the labels they were given. Sub-op 2 stops the track (notes cut, read cursor back to the start), sub-op 3 pauses it (notes keyed off, cursor kept) and sub-op 4 replays it from its first event - the libsnd play call under it resets the cursor before it plays. The engine routes 2 and 3 as a pause and 4 as a rewind-and-resume; it once read 3 as a resume and 4 as a stop, which silenced the score where retail brings it back.
• Movies. Retail's movie player never touches the sequencer; a mid-game movie inherits whatever the script left, and a capture shows a score the script left sounding keeps playing under the movie's audio, and one already stopped (taiku's own stop word, garmel after its boss fight) stays silent. Only the title attract acts: it releases the title theme, and the card-init screen it returns to streams the theme again from the top. Both of the port's hosts follow one engine policy - retail by default, so a cutscene movie layers over the score; the attract stops and restarts the theme; an optional ducking policy lowers only a sounding score and gives back only what that movie took.
• Entry-script pauses. town01 starts id 2016 and, while a system flag is clear, pauses it at once - the opening's silent dawn, restarted by the cutscene's own sub-op 9. The engine's free-roam staging drops that pause on scene-picker entries only; the new-game chain keeps it.
• Battle swap (engine). World::set_battle_bgm queues an ordinary sub-op 1 start on encounter and resumes the field track on battle end. Default battle track: global id 2026 (retail's battle_id == 0 bundle, sound-test #26). Retail BGM changes are hard cuts, so AudioOut::swap_bgm key-offs the outgoing sequencer (notes release through their own envelopes) and starts the new one that instant, with a couple of frames of click-guard on the master.
• Minigame pages take the same path: Baka Fighter loads extraction 1043 (#55), the dance overlay 1048 / 1054 (#60 / #66); the slot machine, fishing and Muscle Dome inherit their host scene's track.

Battle voices: shouts and the melee grunt

When Vahn lands a Somersault he shouts. That shout is not an SPU sample - it is a short CD-XA clip streamed off the disc, picked at random from a small per-art channel pool so it does not repeat back to back. The plain melee swing works the same way, which is why a port that wires up only the SPU stays silent in a fight.

Engine. Executing an art pushes one BattleShoutCue; boot demuxes XA2 / XA4 / XA6 from the raw 2352-byte sectors (disc-image boots only); AudioBgmDirector::play_art_shout resolves the pool pick and AudioOut::play_xa_shout mixes the PCM the way the CD input does, never through the 24 voices. Two timing behaviours model the CD contract: a ~150 ms response delay (the shout trails the animation) and back-to-back requests queue rather than cut. The cue tables are parsed by legaia_art::arts_voice.

Shipped. The melee grunt and the impact cue are staged: boot demuxes the XA27 / XA30 clip banks (read_battle_xa_clip_bank) and the live battle loop fires the per-strike cue (World::fire_melee_impact_cue) - the XA30 grunt on the strike that commits the target's +0x1F3 reaction (four captured ordinary swings emit neither cue), the 0x10C sting only when the effect handle _DAT_8007BD84 is non-null, the s7 pose gate honoured as retail's per-strike condition.

Sony's sequencer and SPU driver

The music player is Sony's, not Legaia's. libsnd (0x80061B18..0x800681D8) walks the SEQ bytes, keeps per-channel state in 16 slots of 0xB0-byte records, and turns note events into key-ons; Legaia installs two function-pointer hooks to wire the script VM's sequence context in. libspu (0x80068..0x8006D) pokes the chip's registers, runs the four-mode DMA transfer engine and the block allocator behind SsSpuMalloc / SpuFree. Being the same libraries every PsyQ game shipped is what lets the functions be named.

• Key-on volume chain. Velocity × bank volume × program volume × tone volume widened into the 14-bit register domain, a channel-volume fold, one pan attenuation per source, and - on the sequencer path only - a closing square taper. Engine port VabBank::fire + sequencer::channel_mix.
• Pitch law. A note is pitched against its tone's center through a two-octave table into a 14-bit SPU PITCH (clamped 0x3FFF).
• Bank open validates the pBAV magic, builds the program-to-tone-page rank map into the reserved ProgAtr words, allocates SPU memory and stashes per-VAG addresses. Engine port VabBank::upload.
• Tempo becomes an integer tick step where the wall clock meets the sequencer; the decoder does not consume a whole event per call.
• Not audio: the 0x801CE628 cluster next door is libpad. And none of this is libgpu - battle and field code fire cues during render passes, which is why the cluster looks renderer-adjacent in call graphs.

Engine port

crates/engine-audio ports the stack as a from-scratch model from this page, the libspu API surface and the public SPU register layout. No Sony bytes.

• Reverb is Studio C, always on, on every voice. Every captured state - field, town, battle, summon, title, minigames - has the master enable set and the 32 reverb registers byte-identical to the StudioC preset. A per-vsync capture reads the per-voice enable register as all 24 bits set on every frame, and the output depth, which is not part of any preset, as a fixed pair. The engine calls Spu::set_retail_reverb() once at init and installs the measured depth. No SEQ carries a CC91 reverb-depth event.
• Timebase. The production path ticks the sequencer once per SPU sample with an exact integer accumulator - locked to the audio clock, no drift, bit-deterministic for the replay oracle. The SEQ header tempo is a 240 BPM placeholder overridden by the first body tempo event, which carries no MIDI length byte; mis-parsing it pins playback ~3x fast (seq).
• Not modelled: pitch modulation, noise and FM (Legaia uses none), asynchronous DMA timing, and the hardware's 39-tap reverb FIR (decimation + zero-order hold instead).

XA-ADPCM Confirmed

Cutscene audio and voice clips are streamed CD-XA: the drive plays them directly, interleaved with video sectors, without the CPU or the SPU voices in the loop. crates/xa decodes the 4-bit ADPCM bit-exactly against an external lossless reference; the demuxer splits raw 2352-byte sectors by (file, channel), which a 2048-byte ISO view strips. Format on xa.

Checking the port: the audio-trace oracle

The port's sound is checked by comparing which SPU voices are active, frame by frame, against a real emulator running the retail disc - the audio sibling of the VRAM and mode-trace oracles. Two retail capture shapes emit the same AudioTraceFrame JSONL: a single-cycle snapshot lifted from a mednafen save state's SPU section, and a multi-frame trace from autorun_audio_trace.lua in PCSX-Redux. The engine runs a standalone SPU + sequencer beside a headless boot session; a retail frame converges when some engine frame's active-voice mask is a superset of retail's with every retail voice's start address matching. A start-address or master-volume mismatch is a hard failure, and so is an engine trace that sounded nothing at all where retail had voices.

What a save-state comparand cannot decide is the rest. An SPU voice leaves its off phase on key-on and returns only when a key-off's release runs all the way down, and nothing does that wholesale - so a save taken mid-playthrough reports nearly every voice non-off, the residue of every cue since boot rather than the set the current score is sounding. A cold scene-entry window holds single-digit concurrent voices, so it can never be a superset of that, however faithful the playback. Deciding which voices belong to the score needs the per-vsync PCSX-Redux trace, captured from a moment whose loaded track is the one the engine side plays - which the save's scene does not settle.

That comparand once read as "the engine is much the quieter side", with reverb routing as the first channel to differ and a roughly halved voice count behind it. Both were instrument defects. The reverb figure came from the wrong register: the emulator's SPU register blob is the hardware window verbatim, and the offset being read was the SPU control register, not the per-voice reverb-enable mask. Read correctly, retail routes all 24 voices, and so does the engine - the oracle had been building a bare SPU where the shipped audio host builds one through set_retail_reverb(), so it was measuring an engine the port does not ship. Reverb now agrees frame for frame on routing, depth and work area.

The voice-count gap was two different pieces of music. An engine trace plays the track the scene's own entry script selects; a retail save plays whatever the playthrough left loaded, and a town save reached by walking in from the world map still holds the overworld track. Paired on one track and run over a comparable stretch of it, the two sides hold the same number of concurrent voices. The rest of the old gap is where in the track each window sits: that track's own score, decoded straight from the sequence data, peaks at nine simultaneous notes in its first few seconds and nineteen over the whole piece - and nineteen is exactly what the retail capture shows. With trace logging on, the sequencer drops no notes at all across the window.

What is left after track and window are controlled for is one per-voice difference: retail spends slightly more voices than it has distinct notes, doubling a note across two slots about one time in six, where the port plays each note once. The trace record now carries per-voice envelope level, volume and envelope-config word from all three emitters, so that question can be asked of the artifact directly.

CLI: legaia-engine audio-trace --scene NAME with --scenario (snapshot) or --retail-jsonl (multi-frame), plus --per-voice for the allocation comparison. Tests audio_trace.rs, audio_trace_multi.rs (needs LEGAIA_AUDIO_TRACE_JSONL_DIR).

How we know

What's left

• The bse.dat battle bank: header word, record stride and the record columns (the SFX-table columns; +4 is the category byte the cue router FUN_8004FE5C rewrites per cue through gp+0x678) are pinned (sound-driver outputs).
• The melee grunt and impact cue in the engine: an XA30 bank at boot and cue emission from the live battle loop.
• The .MAP / .PCH PsyQ SoundArtist intermediates are dev-side inputs, not separate retail chunks; nothing to port.

See also