Audio

How it works

The PSX has a dedicated sound chip (the SPU, Sound Processing Unit) with its own 512 KB of memory. Games can't talk to the SPU directly from main RAM - they use Sony's PsyQ libraries (libsnd for sequencer playback, libspu for low-level voice control) which speak the SPU's wire protocol. Legaia links both statically, so the libraries are part of the game binary and can be inspected like any other code. That inspection is how this page was written: we traced the original code with a disassembler (Ghidra), and the FUN_80xxxxxx names throughout are the RAM addresses of functions in that trace.

PSX games typically split audio into three concerns: sound effects (short one-shot samples), BGM / music (sequence-driven, longer - a SEQ is a MIDI-like note stream played through a VAB instrument bank), and streaming audio (XA - compressed audio the drive plays straight off the disc - and .pac voice). Legaia is no exception, but its plumbing is layered:

• A path-string cluster at 0x8007B380 holds the file extensions the sound subsystem appends to scene-asset paths: .spk, .LZS, .dpk, .MAP, .PCH, .pac, STR, plus the master file bse.dat.
• Two consumers in SCUS_942.54 (the game's main executable) turn those paths into actual loads: a sound-init / .dpk loader and a streaming-asset loader for XA-family files.
• Underneath that is Sony's PsyQ SsAPI sequencer (libsnd, statically linked) for SEQ playback, plus the PsyQ libspu primitives that actually move bytes into SPU RAM.

The fully-documented formats on disc are the VAB sound bank (Sony's standard VABp-magic instrument bank) and the SEQ sequence. The per-scene .dpk / sound_data2 pack is decoded as a VAB + SEQ bundle (type-0 VAB header + type-1 VAB samples + type-2 SEQ); the .MAP / .PCH / .spk / .pac PsyQ intermediates are dev-side inputs, not separate retail chunks here.

BGM dispatch is genuinely surprising: there's no literal "BGM ID → file" table. A field-VM script writes a BGM ID, and a per-frame poller resolves it to an entry number in PROT.DAT (the disc's single big archive) using the current scene's CDNAME block layout. So "the table" isn't in the executable - it's in the on-disc archive layout itself.

Path-string cluster

The string cluster at 0x8007B380 holds the file extensions the sound subsystem appends to scene-asset paths. Eight extensions: .spk, .LZS, .dpk, .MAP, .PCH, .pac, STR, bse.dat (master file). Full layout in the sound-driver format spec.

SCUS consumers

FUN_8001EBEC looks like a third "mode-aware extension dispatcher" but is not a sound consumer: the decomp shows it is the graphics-side character-TMD equipment-conditional group-transform swap (it reads DAT_8007C018[_DAT_8007B824 + 0..2], the loaded battle-character TMD pointers). See character-mesh.

Both FUN_8001FA88 and FUN_8001FC00 carry a dev/retail split via _DAT_8007B8C2. Retail is the != 0 arm and loads by PROT index directly; the == 0 arm is the dev path, opening an h:\ path through FUN_8003E6BC. That function does no CDNAME name resolution - its body is strcpy → break 0x103 (a PsyQ dev-station host trap) → fseek/fread/fclose, on paths absent from a retail disc.

VAB sound banks

Sony's standard VABp-magic instrument bank format, documented at formats/vab. The dominant on-disc carrier is the scene-VAB-prefixed streaming shape - the VAB body is preceded by a 4-byte chunk0 header. Implementation: crates/vab (header parser + extractor + ADPCM decoder).

Bulk scan finds 1191 VABp headers across 239 PROT entries. Multi-bank archives at 0889_sound_data2, 0890_sound_data2, 0891_level_up. The vab_01 cluster (CDNAME indices 1072–1194) is the standard distributed-bank layout.

Per-actor sound effects

Function

FUN_800250D4(sound_id, voice)

Called from

The actor tick (FUN_80021DF4) when actor[+0xb4] != 0 (one-shot pulse) or actor[+0xac] is staged (continuous)

Looks up a sound entry at &DAT_8006F198 + sound_id*8 for sound_id < 0x200, or in the runtime-allocated table at _DAT_8007B8D0 for higher IDs (the .dpk consumer's bank). The entry's byte[3] & 0x1F is the voice count; the helper then calls FUN_800653C8 (libSPU SpuKeyOn-equivalent) for each of voice..voice+count-1.

Move-VM and field-VM opcodes write actor[+0xac] (sound ID) and actor[+0xb0] (voice); the move-VM tick re-fires the SFX whenever the trigger flag at actor[+0xb4] is set.

Monster sound bank - h:\mpack\monster.snd

Battle-time monster sound banks live in a single packed monster.snd file. The loader is FUN_8003E104(monster_idx, slot, dst_buf) - called twice from the battle scene loader FUN_800520F0 (slots 7 and 8, for the active battle's two monster sound banks). It reads the file's per-monster TOC at 0x801C8980 - 0x10 (4-byte stride, paired entries giving [start_lba, end_lba+1]), computes the LBA range, and dispatches:

• Dev path (_DAT_8007B8C2 == 0) - the host-trap file API: FUN_800608F0 → FUN_80060920 (fseek to record × 0x800) → FUN_80060944 → FUN_80060910. Path string: h:\mpack\monster.snd. Retail never takes this arm.
• Retail path - stages (size, dst) into the gp window at +0x97c / +0x894, kicks the async CD read via FUN_8003F128. Sets a 120-frame timeout at +0x91c.

The same pattern (h:\mpack\… paths + per-record TOC at a small data structure) recurs across the audio loaders - the FUN_8003E104 dump is the canonical example.

BGM dispatch

The field VM's opcode 0x35 writes the BGM ID to _DAT_8007BAC8. FUN_800243F0 (the per-frame asset poller) resolves it to a PROT index - bgm_id < 2000 is scene-local, bgm_id ≥ 2000 is a global pool. There's no literal BGM table; the resolution is a PROT-relative offset into the CDNAME per-scene block.

See script VM → "BGM lookup table" for the resolver code.

The engine port reuses this same dispatch for the Battle↔Field music swap: World::set_battle_bgm configures a battle track id, and the live gameplay loop queues an ordinary FieldEvent::Bgm{sub_op: 1} start for it on encounter (swap_to_battle_bgm) and resumes the stashed field track on battle end (restore_field_bgm). The host's AudioBgmDirector cross-fades both transitions over ~0.5 s through its existing start_inner path - no separate battle-audio code path. The battle id resolves like any op-0x35 id: scene-local (< 2000) through the scene's own BGM table, global-pool (>= 2000) through the music_01 bank arm. The shipped default is the global id 2026 (retail's standard battle theme), installed by LiveLoopOpts::playable() so both hosts swap without per-scene configuration.

SsAPI sequencer - the MIDI-style music player (0x80061-0x80067 cluster)

Legaia statically links Sony's PsyQ libsnd / SsAPI sequencer for .SEQ-driven music - the layer that walks the sequence bytes, tracks per-channel state, and turns note events into SPU voice key-ons. The cluster lives in SCUS at 0x80061B18..0x800681D8 and uses the standard SsAPI globals: a 16-slot allocation bitmap, per-slot 0xB0-byte sequence-state structs, and two Legaia-installed function-pointer hooks (_DAT_801CE564/_574) that wire the script VM's seq context in. The full symbol catalogs are in the disclosures below.

None of these functions is libgpu / libgs - they are all libsnd. Battle / field code triggers SFX cues during render passes, which makes the cluster (FUN_80061EDC / FUN_80067E9C / FUN_80066E50 / FUN_80067550) look renderer-adjacent in call graphs; don't file it under the renderer.

Interpretation: _DAT_8007BAC8 = bgm_id written by field-VM 0x35 is consumed by FUN_800243F0 to load a .SEQ payload via the streaming-asset path, and that payload is then handed to FUN_80062340 for sequencer playback.

Engine reimpl can stub the entire cluster behind a legaia-engine-audio::Sequencer trait without touching the per-note math.

libspu / SPU control (0x80068-0x8006D cluster)

Sits underneath the SsAPI sequencer and drives the SPU hardware directly - PsyQ libspu, statically linked. It owns the SPU MMIO register base, the SPU-RAM block allocator, and the mask-driven attribute setters that write per-voice and common registers. Symbol catalogs below.

SPU DMA transfer engine

Sits between the SsApi seq layer and the libspu register primitives. This is the path SEQ/VAG bytes take when moving from PSX RAM into SPU RAM: a 4-mode transfer state machine, a top-level SpuWrite that picks DMA vs CPU copies, and the block allocator/compactor pair behind SsSpuMalloc/SpuFree.

SsApi seq-management layer (above libspu)

The bank/track bookkeeping layer: VAB open/close by id, the VAB-open head + program-change pair, and the status/callback accessors the game polls. Chain: slot bitmap @ _DAT_801CD2B8 → ptr table @ 0x801CD2C0 → per-slot record (stride 0x36) at 0x801CDB60 → VAB program-attr (stride 0xB0).

File-API leaf cluster

The dev/retail split for sound + monster-bank loading routes the dev branch through libapi-style file primitives at FUN_800608E0..FUN_80060A04: fopen / fseek / fread / fclose plus a vsync_wait (FUN_8005FCCC) and a BREAK 0x105 trap at FUN_80060A04. These are PsyQ kernel-call wrappers around the BIOS A() table - FUN_80056738 / 80056748 / 80056768 / 80057014 / 8005ACE8 are all jr 0xA0 BIOS dispatchers.

Engine reimpl can map the entire cluster to std::fs + a frame-paced sleep.

Engine-audio model - clean-room SPU port

crates/engine-audio ports the SPU side of the audio stack as a clean-room model. No Sony bytes; the spec is this page plus the libspu API surface and the standard PSX SPU register layout.

What this does not model (out of scope for the first port pass):

• Pitch modulation, noise, FM. None used by Legaia (verified against the libspu calls in the SCUS dumps - SpuSetPitch is the only pitch path).
• Asynchronous DMA timing. The transfer engine here is synchronous (queue + drain collapsed) - fine because the playback layer reads SPU RAM directly during voice ticks. The model preserves the API shape (set_transfer_start_units_8 / set_direction / write) so the libspu callers map cleanly.

Reverb model (engine-audio)

The retail SPU implements reverb as a same-side / different-side IIR reflection pair feeding a 4-tap comb early-echo and two all-pass stages, run at 22050 Hz over a work buffer at the top of SPU RAM (mBASE = 0x80000 - work_size). The 9 standard libspu modes (Room / StudioA-C / Hall / Space / Echo / Delay / Pipe) plus Off each select a 32-register set (work-area size + IIR/comb/all-pass coefficients + tap addresses).

The engine-audio clean-room port reproduces that network register-for-register in spu::reverb: each ReverbMode loads the standard libspu preset (public PSX hardware-reference constants - the same tables every open SPU emulator ships, not Sony game data) into a recirculating i16 work buffer sized to that mode's work area. Address registers are in 8-byte units, taps wrap within the work area, and the reverb multiply is (sample × coeff) / 0x8000 (signed Q15, so a 0x8000 coefficient inverts phase exactly as the hardware does). So the wet tail carries the real comb/all-pass colouration the retail modes produce, not a flat slap-back.

Per-voice routing is opt-in: Voice::reverb_send = true (libspu SpuSetVoiceReverb analogue) sums the voice's pre-master output into the reverb send bus; the wet output is mixed back into the master in Spu::tick.

Retail reverb routing - Studio C, always on

Retail runs one global reverb configuration everywhere:

• The reverb network is master-enabled in every captured state (SPUCNT bit 7) - field, town, battle, summon, title, minigames. No scene toggles it.
• The mode is Studio C everywhere. The 32 reverb registers (0x1F801DC0..) are byte-identical across all captured states and match the StudioC libspu preset exactly (dAPF1=0x00E3, work area 0x6FE0); ReverbMode::identify resolves the captured block to StudioC.
• Per-voice reverb-send (EON) is broad - typically 15–22 of 24 voices, BGM + SFX alike. Reverb is the default routing, not a per-cue effect.

There is no per-cue reverb-enable source: the live engine matches retail by calling Spu::set_retail_reverb() once at SPU init - ReverbMode::StudioC + every voice routed. Pinned by a pure-Rust sweep of the save-state corpus (SPU register shadow via PsxSpu::reverb_registers / voice_reverb_mask / reverb_master_enabled; CLI mednafen-state spu).

A "Spirit-Arts / echo cues selectively opt into reverb" reading (which would imply a per-cue reverb-enable source to trace) is falsified by the corpus sweep; the SEQ controller census also finds no CC91 (reverb depth) events, so the stream encodes no per-cue reverb at all.

Boundaries: mode selection via Spu::write_reverb_mode_byte(raw) matches the libspu byte API (1=Room, 2=StudioA, …, 9=Pipe; out-of-range falls back to Off) - the engine half of SpuSetReverbModeParam (FUN_8006B1B4). The hardware's 39-tap FIR input/output resampler (44.1 kHz ↔ 22.05 kHz) is approximated by decimation + zero-order hold, and output volume (vLOUT/vROUT, not part of the mode preset) uses a fixed depth overridable via Reverb::set_output_volume.

Engine-audio model - Sequencer port

The legaia-engine-audio::Sequencer is the runtime side of the SsAPI sequencer cluster above. Surface mirrors SsSeqOpen / SsSeqPlay / SsSeqClose / SsSeqSetVol without copying any Sony bytes:

Voice allocation is round-robin over the 24 SPU voices, with the sequencer tracking (channel, key) → voice so the matching key-off can shut down the right slot. Tempo events from the SEQ override the running tempo at the event's absolute tick (matching libsnd's mid-stream 0xFF 0x51).

Pitch bend (0xEn) is acted on: a bend sets the channel's 14-bit wheel, re-pitches sounding voices, and folds into subsequent NoteOns; the bend range is a per-tone disc value (each VAB tone's pbmin/pbmax semitones, applied exactly as libsnd does - a (0, 0) tone ignores the wheel). Aftertouch (0xDn/0xAn) is parsed, but the retail score never emits it.

Controller census. A disc-wide sweep of every SEQ-bearing PROT entry fixes what the retail score emits: CC7 (volume) and CC10 (pan) carry the bulk; CC99 carries only the two loop-marker values; CC11 (expression) and CC91 (reverb depth) are absent - consistent with reverb being a fixed global.

WASM output. The audio-webaudio feature adds WebAudioOut (ScriptProcessorNode-based) so the same SPU + sequencer stack drives browser targets.

Loop points are read from the stream: the NRPN-style control changes on 0xB0 (controller 99 value 20 = Loop Start, value 30 = Loop Forever). A Loop Start records the position immediately after the marker; a later Loop Forever - or an end-of-track that follows a Loop Start - rewinds there rather than to event 0, so looped BGM repeats from the correct bar instead of restarting the whole track. The rewind resets the integer sample-clock, so the looped body re-fires on the same sample offset every pass. set_loop_to is the fallback for the four retail tracks with no markers.

Timebase. The production playback path ticks the sequencer once per SPU sample (tick_sample), so the music clock is locked to the audio clock. Timing is computed with an exact integer accumulator (units of sample × ppqn × 1_000_000; an event of delta d fires when the accumulator reaches d × tempo_us × 44100) - no per-tick float, no long-track drift, and bit-deterministic for the replay oracle. Note the SEQ tempo gotcha: the header tempo is a 240 BPM placeholder, immediately overridden by the first body 0xFF 0x51 (which, in PSX SEQ, carries its 3 tempo bytes with no MIDI length prefix). Mis-parsing that override pins playback at the placeholder rate (~3x too fast).

Engine-audio model - SFX bank + scheduler

Maps battle / field cue IDs (the kind byte the art-record HitCue / overlay scripts emit) to per-cue SfxEntry descriptors that describe how to fire a one-shot through the SPU. Engines populate the catalog at startup, then forward ScheduledCue-like requests through SfxScheduler which queues each request with its retail timing offset and dispatches when the per-frame tick reaches the firing frame.

SfxBank::from_descriptors builds the catalog from the disc-decoded static SFX table (DAT_8006F198). The bank those programs index is named by the cue itself: FUN_80065034 calls FUN_80068b98(vab_id, program) before the program lookup, and that repoints the libsnd current-bank globals (_DAT_801ce33c/_DAT_801ce334/_DAT_801ce340) at the VAB slot the descriptor's +4 category selects. The older reading - "the globals hold whichever bank is open, so a cue plays out of the scene's music VAB" - was a save-state artefact: those globals are shared with the sequencer, so a state sampled after a BGM note holds the music bank (13 distinct ones across the save catalogue). The engine therefore stages the two pinned SFX banks itself and fires each cue with SfxBank::play_one_shot(spu, vab) against the one its category names, falling back to the scene VabBank only when nothing staged.

SfxBank::play_one_shot delegates to the existing VabBank::play_note for tone lookup, pitch math, and ADSR setup; the scheduler is a frame-driven queue that returns an SfxFireBatch per tick_frame call so engines can dispatch through the same VabBank they already wired for the BGM sequencer. A PendingCue with frames_remaining = 0 fires on the next tick, so a cue queued mid-frame doesn't fire immediately and gives the host a chance to clear render state first - matching the retail timing where a HitCue::timing_frames = 1 cue plays one frame after the strike begins.

Implementation: crates/engine-audio/src/sfx.rs.

XA-ADPCM Confirmed

crates/xa decodes CD-XA 4-bit ADPCM bit-exactly: on a real cutscene track its per-channel PCM matches an external lossless reference decode sample-for-sample. The on-disc .XA / .STR audio is standard CD-XA Mode 2 Form 2 - the earlier “non-standard interleave / ~90% of groups fail validation” reading was Form-1 truncation damage in the old extractor, not a bespoke format. The demuxer (legaia_xa::demux) splits raw 2352-byte sectors by (file_no, ch_no) and the group decoder reconstructs each channel. See formats/xa and cutscene / STR for the interleaved A/V path.

Audio-trace parity oracle

Mirror of the VRAM-byte and mode-trace parity oracles on a third axis: per-frame voice activity. The retail side has two capture shapes, both emitting the same AudioTraceFrame JSONL wire format:

1. Single-cycle snapshot lifted from a mednafen save state's SPU section via legaia_mednafen::PsxSpu (24 voice records, master volume sweep, voice-on/-off masks, reverb mode, 512 KiB SPU RAM). One .mc{slot} save → one retail AudioTraceFrame. Convergence: "did any engine frame in the window match retail's voice mask?".
2. Multi-frame trace captured by autorun_audio_trace.lua running inside PCSX-Redux: per-vsync PCSX.createSaveState() calls, the SPU sub-message sliced out via FFI pointer arithmetic, decoded offline into JSONL by extract_audio_trace_from_sstates.py. Convergence becomes "for every retail vsync with audio playing, did the engine ever match?", applied frame-by-frame via first_audio_trace_divergence_multi.

The engine side runs a standalone legaia_engine_audio::Spu + optional Sequencer alongside a headless BootSession::tick, sampling voice / master / reverb state after each frame. JSONL is one AudioTraceFrame { frame, sequencer_playhead_ticks, sequencer_finished, master_volume, reverb_mode, active_voice_mask, voices[24] } per line. Convergence rule per retail frame: at least one engine frame's active_voice_mask is a superset of retail's mask AND for every retail-active voice the engine matches start_addr (when both sides report it).

PCSX-Redux's Lua API does not expose the SPU register file directly (SPUInterface::lockSPURAM is C++-internal, not bound). The probe leans on PCSX.createSaveState(), which returns the full state as a protobuf slice (~20 MiB); the autorun script walks the slice in-place via FFI and writes only the ~600 KiB SPU sub-message to disk so per-vsync GC pressure doesn't disrupt GPU::Vsync event delivery. The SPU schema is the one declared in PCSX-Redux's src/core/sstate.h + src/spu/types.h: Channel.Data.on || .stop is the retail-side "audible" criterion (ADSRInfoEx.state reads as Sustain even for unused voices, so it's not a reliable audibility signal).

Two known asymmetries the diff function explicitly models:

1. Headless engine SPU. BootSession only attaches a real cpal AudioOut when enable_audio = true, which fails in CI. The oracle constructs a standalone Spu in parallel and routes scene-resolved BGM events into it. Not bit-identical to the retail SPU, but the voice-activity envelope is.
2. Retail capture shape. The single-snapshot case freezes one SPU cycle; the multi-frame case carries per-vsync state. The engine produces frames + 1 records either way. NoFrameMatched stays tolerable drift in both modes; VoiceStartAddrMismatch and MasterVolumeMismatch are hard failures.

The engine drives BGM through a private TraceBgmDirector that routes field-VM op 0x35 events into a headless Sequencer in lock-step with SceneHost::route_bgm_events - so the engine trace does play music. NoFrameMatched is treated as tolerable drift (the scene prescript may not emit op 0x35 within the trace window, or may target a different track than retail captured); VoiceStartAddrMismatch and MasterVolumeMismatch are hard failures.

Entry points:

• Library: engine_shell::audio_trace_oracle - build_engine_audio_trace, load_runtime_audio_trace_from_save, load_runtime_audio_trace_jsonl, first_audio_trace_divergence, first_audio_trace_divergence_multi, JSONL round-trip.
• CLI: legaia-engine audio-trace --scene NAME (explicit), --scenario LABEL (single-snapshot vs .mc{slot} SPU), or --retail-jsonl PATH (multi-frame vs PCSX-Redux capture).
• Disc-gated tests: audio_trace.rs auto-discovers scenarios with both expected_active_scene and an on-disk .mc{slot} save; audio_trace_multi.rs walks the same scenarios but skips unless LEGAIA_AUDIO_TRACE_JSONL_DIR points at a directory of <label>.jsonl files from the PCSX-Redux probe.

What's left

The per-scene .dpk / sound_data2 pack is decoded: it's a VAB + SEQ bundle (type-0 VAB header + type-1 VAB samples reconstitute one VAB; type-2 = SEQ), parsed by legaia_asset::sound_pack. The bse.dat master bank header is also pinned (the +2 ushort is an even byte-offset splitting it into two sections, not a record-count divisor). Residual: the .MAP / .PCH PsyQ SoundArtist intermediates are dev-side inputs that produce these VABs, not separate on-disc retail chunks.

See also