DATA_FIELD streaming format Confirmed

At a glance

Magic

none - the first u32 is already a chunk header (type byte in the top 8 bits)

Where on disc

34 PROT.DAT entries strict-validate as class data_field_streaming: the other5 character/effect run (1204..1219) and a family of scene bundles

Parser

crates/container-formats/src/lib.rs::parse_streaming; asset scan-stream / asset extract

Confidence

Confirmed - consumer FUN_8002541C, the 0x14 (DATA_FIELD) branch, feeding FUN_8001F05C with copy_only=1

Used by

asset loader, battle scene loader, characters page (battle-form meshes + atlases)

Overview

The walker passes every chunk through the dispatcher with copy_only=1, so chunks are always uncompressed - the container trades disc space for a load path with no decompression step. Implementation: crates/container-formats/src/lib.rs::parse_streaming.

Layout

A stream is a chain of chunks with no count up front - each header says how far to the next one:

Every header is the dispatcher's own packed word, stored little-endian:

What's in the wild

asset scan-stream strict-validates 34 PROT entries (class data_field_streaming), in two families.

Character / effect streams

The other5 run, entries 1204..1219. The common shape is three single-asset chunks:

The two heads of the run are homogeneous instead: 1204 is five TMD2 chunks (the battle-form party meshes used by the Baka Fighter minigame and as the default-equipment sibling pack - the in-battle player meshes themselves are assembled from the PLAYER1..4 files) and 1205 is eight TIM chunks of 0x8220 bytes each (their atlases) - which is what makes the atlas stride 0x8224, chunk header included.

Scene bundles

MAN / MES / MOVE / VDF, four chunks (dolk2, rikuroa, rikuroa2, rayman, station, taiku, taiku2, doman, nilboa2, edbalden, eddoman). Shorter variants drop the trailing VDF (balden2, ropeway2) or lead with a bare TMD where a MAN would sit (balden, bubu1, edbubu). Two entries are neither family: init_data is TIM_LIST + TMD, and 0890_sound_data2 is two TIMs.

The chunk layouts are single assets in the other5 family (one TIM, one TMD2, one MOVE2), not packs. Other clusters do use pack-shaped TIM_LIST / TMD chunks; the pack format handles that case.

Trailer data

Every clean stream ends inside the last 2048-byte sector of its entry, and the bytes from the terminator to the sector boundary are what asset extract keeps as _trailer.bin. The walker stops at the terminator and reads none of them. They are sector fill: across all 34 streams the trailer is shorter than a sector and zero, except in entry 1208, whose fill carries eight stray bytes (78 FD 00 00 01 00 0D 00 at 0xFD70) that nothing is known to read.

Related shapes

• The scene-TMD-prefixed streaming shape is structurally similar but the leading chunk has no [u32 type_size] header and the inner content is a bare TMD instead of a TIM.
• The scene-VAB-prefixed streaming shape uses the same chunk0-header trick but with VAB content.
• Pack format lives inside TIM_LIST / TMD chunks when the chunk's data is a pack rather than a single asset.
• One entry (extraction 0892) matches the data_field_truncated detector - three clean leading chunks, then an over-large fourth header - and the match is an artefact. The entry is an container-formats::pack (count = 2, word offsets [3, 0x208B]), and those "chunks" are the pack's own header words read as chunk headers. Mode-22 CARD INIT (FUN_8002574C) walks it as a pack and uploads each member through FUN_800198E0. See the card font pack.

Entry 0892 (card_data) is a pack, not a truncated stream

Extraction entry 0892 is a two-member container-formats::pack of 4bpp TIMs, loaded once per memory-card-screen entry. The "12 MB LZS container" figure that once stood against it came from the superseded declared span (toc[p+5] - toc[p+3] + 4 = 5977 sectors); the entry is 33 sectors, 67,584 bytes.

What the runtime does

The entry-context selector is gp+0x7E8 (0x8007BB00): 0 skips the block, 1 takes the disc load, and any other value takes a third arm that MoveImages the same VRAM footprint back from a parked copy. Every writer of the slot stores 1 and the menu overlay clears it to 0, so no retail path reaches that third arm - but its inverse is live: the menu overlay parks (0,475) 256×1 to (0,492) and (320,256) 128×256 to (704,0) on the way into game mode 0x1A.

The two members

Together they tile VRAM (320..447, 256..511) - exactly the rect both MoveImage pairs move. The 948 bytes past member 1 are unreferenced tail slack.

The content: a JIS X 0208 level-1 kanji font

The CLUT block is identical in both members and is not an ordinary palette bank. Rows 0..3 hold the index patterns 0101…, 0011…, 00001111… and 00000000 11111111 against one ink, and rows 4..7 repeat them against a darker ink - each row selects one bit of the 4bpp index. The texture is a 1bpp bitmap packed four bit-planes deep, and a draw picks its plane by CLUT row. Rows 8..15 are ordinary 16-colour palettes for other content on those rows.

Per plane the grid is a 12-pixel pitch with an 11×11 ink box: columns and rows 11, 23, …, 239 carry no ink in any plane and nothing is inked past 238. That is 20×20 = 400 cells per plane, 4 planes per member, 8 planes in all - seven full and the last inked through cell 164, for 2965 inked cells, exactly the JIS X 0208 level-1 kanji count. Plane 0 reads in level-1 ku-ten order, so a glyph's cell is plane = g / 400, row = (g % 400) / 20, col = g % 20.

Which routine samples the page? Nothing does: the page is loaded, parked and never drawn in this build. A GPU-FIFO watch at card-screen entry (mode 0x16 → 0x17) sees no primitive sample it (settled), and the static evidence agrees. A sampler would have to carry two constants, and the members' headers pin both: 4bpp pages at (320, 256) and (384, 256) give tpage ids 0x15 / 0x16, and picking a bit-plane by CLUT row gives a CBA of 0x76C0 + plane × 0x40 (rows 475..482). Neither appears on the disc: sweeping SCUS_942.54 and all 83 statically based overlay images for every one of the eight CLUT ids, as an instruction immediate and as a bare halfword (what a sprite-descriptor entry would be), finds only incidental instruction halves, and the only two 475 immediates disc-wide are the two MoveImage park/restore sites. The uploader leaves no handle behind either. The live path parks the page into the world-map texture region (704, 0) on the way out, and the save / load screen's own text is drawn from the dialog font page at (896, 0). The upload still runs on every mode-22 entry.

Per-scene field bundles

The CDNAME block for a typical field/town scene (e.g. town01, bubu1) carries 8-12 PROT entries. Categorize (the sweep that classifies every archive entry by format) identifies several known shapes per block. In the retail block frame (see CDNAME § numbering space) the head of every scene block is fixed: slot 0 the scene's .MAP, slot 1 its v12 trigger sidecar, slot 2 the event prescript, slot 3 the scene asset table. The table below is the categorize sweep's per-class view, whose slot columns predate that frame and are approximate.

Placed actors use their own scene's models. A placement's model byte below 0xF0 indexes the scene's TMD list, and one at or above it a global head slot (the player character pack). Across the 101 scene blocks with a MAN, every placement below 0xF0 resolves into its own block. The only exceptions are 20 placements in other7, an unused draft of koin3, whose model bytes run past its own TMD list.

The categorize sweep classifies every PROT entry to something; the items above are the residual edge. See the engine page for how scene loading consumes these shapes.

How we know

Source of record: docs/formats/data-field.md.

See also