Cutscene (STR mode)

Game modes

StrInit bootstraps the cutscene: opens the STR stream, initialises the MDEC decoder, starts the XA channel. StrMode runs the per-frame loop: reads the next batch of sectors, decodes a frame, blits it full-screen, and advances the audio position. When the stream ends, the mode chain transitions to ConfigInit (index 1).

The retail handler is overlay-resident (cutscene overlay, not yet captured).

STR sector format

STR video is carried in 2048-byte Mode 2 Form 1 sectors. Each sector's user-data area starts with a 20-byte header, followed by 2028 bytes of compressed bitstream payload:

Offset  Bytes  Field
0x000   2      magic            — 0x0160 = video sector; any other value = non-video, skip silently
0x002   2      chunk_number     — 0-indexed position of this sector within the frame
0x004   2      chunks_per_frame — total sectors needed to complete this frame
0x006   2      frame_number     — sequential, 0-based, wraps at 0xFFFF
0x008   4      bs_data_size     — total bitstream bytes across all chunks for this frame
0x00C   2      width            — frame width in pixels (multiple of 16)
0x00E   2      height           — frame height in pixels (multiple of 16)
0x010   2      bs_version       — 2 (BS v2 only in Legaia)
0x012   2      quantize_scale   — per-frame quantization scale, 0..63
0x014   2028   bs_data          — compressed bitstream payload chunk

StrFrameAssembler accumulates sector payloads in arrival order. When chunk_number + 1 == chunks_per_frame, the full bitstream is returned truncated to bs_data_size. Non-video sectors are skipped silently.

Implementation: crates/mdec/src/str_sector.rs

MDEC decoder

MdecDecoder::decode_frame(bs) converts a complete BS v2 bitstream into an RGBA8 pixel buffer. Clean-room port; reference: PSX-SPX §MDEC Decompression.

1. Bitstream header

4 bytes before the macroblock data: u16 n_words (number of 32-bit words) + u16 qs (per-frame quantization scale, 0–63).

2. VLC decoding

Each 8×8 block decodes its DC coefficient first, then AC coefficients until an EOB token.

• DC coefficient — luma and chroma use separate VLC tables (PSX-SPX Tables B.12 / B.13). Delta-coded per component (Cr / Cb / Y0–Y3 each have independent running state).
• AC coefficients — MPEG-1 Table B.14 (run/level pairs). Escape sequences carry a 6-bit run + 8-bit signed level. EOB code (run == 64) ends the block.

After VLC, coefficients are in JPEG zigzag scan order.

3. Dequantize

coef[i] = clamp( (coef[i] * qs * Q_MAT[i] + 4) / 8, -2048, 2047 )

Q_MAT is the standard PSX quantization matrix. The DC position always gets qs = 2 applied independently.

4. 8×8 IDCT

Two-pass separable 2D IDCT using a precomputed cosine table IDCT_C[k][n] (pre-scaled by 2048). Row IDCT followed by column IDCT; output clamped to [-128, 127].

5. Macroblock layout

Macroblocks are 16×16 pixels. Each macroblock decodes 6 × 8×8 blocks: Cr, Cb, Y0 (top-left), Y1 (top-right), Y2 (bottom-left), Y3 (bottom-right).

6. 4:2:0 upsampling + BT.601 colour conversion

Each Cb/Cr sample covers a 2×2 luma region. Chroma values are center-biased (nominal ~128, subtracted before the matrix).

R = Y + ((91881 * Cr) >> 16)
G = Y - ((22554 * Cb + 46802 * Cr) >> 16)
B = Y + ((116130 * Cb) >> 16)
A = 255

Output is a width × height RGBA8 buffer in row-major order.

Implementation: crates/mdec/src/lib.rs (MdecDecoder, VLC tables, IDCT_C, Q_MAT).

XA audio

XA-ADPCM audio is carried on Mode 2 Form 2 sectors with submode & 0x24 == 0x24. The demuxer splits them by (file_no, ch_no) into per-channel streams. Each 128-byte sound group holds 8 sound units of 28 4-bit ADPCM samples; stereo interleaves as SU0 = L, SU1 = R.

See docs/formats/xa.md for the full sector layout, coding-info bit definitions, and filter coefficients.

The mapping from cutscene name to expected (file_no, ch_no) channel pair is overlay-resident (in the not-yet-captured cutscene overlay). Until that's reversed, WAV → cutscene assignment is manual.

Playback loop (play-str)

legaia-engine play-str <file> demonstrates end-to-end decoding:

1. Read the raw file in 2048-byte sectors.
2. Feed each sector to StrFrameAssembler::push_sector().
3. On complete frame: MdecDecoder::new(w, h).decode_frame(&bs) → RGBA8 buffer.
4. Pre-decode all frames, then enter the winit event loop.
5. On RedrawRequested: upload the next frame as a GPU texture and render fullscreen.

# Report frame inventory
mdec scan-str cutscene.str

# Decode all frames to PPM images
mdec decode-str cutscene.str --out-dir frames/

# Play in a window
legaia-engine play-str cutscene.str

Open items

• Cutscene overlay capture. The retail StrInit / StrMode handlers are in a dedicated overlay. Save state during any pre-rendered cutscene and run scripts/analyze-overlay.sh; import as overlay_cutscene in Ghidra. Unblocks: XA channel map, PROT-to-STR entry table, legaia-engine play --scene cutsceneN.
• XA channel map. (file_no, ch_no) → cutscene-name association is inside the cutscene overlay.
• PROT-scene routing. Direct play-str <file> works; routing play --scene cutsceneN through the CDNAME entry is pending the STR-entry trace.

Provenance