Streaming Internet Archive Video Across Two Architectures

The TV Guide Concept

The TV Guide is one of 16 apps in OASIS_OS. It presents five channels, each playing content from the Internet Archive on a deterministic schedule. The scheduling uses a seeded PRNG: the same seed plus the current timestamp always produces the same program at the same playback position. Tune to Channel 3 at 8:42 PM and you'll see the same content every time, on every device.

The challenge: MP4 files from the Internet Archive range from 50MB to 2GB, with the critical moov atom (which contains the track tables, sample sizes, and timing information) placed either at the start or the end of the file. The streaming architecture must handle both cases, on two radically different platforms.

Desktop Architecture: StreamingBuffer

The desktop pipeline uses in-process progressive streaming via a custom StreamingBuffer that implements Read + Seek over an Arc<StreamingInner> sliding-window buffer. A background download thread feeds the buffer while symphonia (audio/video demuxer) reads from it simultaneously.

Desktop Streaming Pipeline

  Download Thread                  Decode Thread
  ─────────────────                ─────────────────
  HTTP GET file.mp4               symphonia probe
        │                               │
        ▼                               ▼
  ┌─────────────────────────────────────────────┐
  │          StreamingInner (Arc)                │
  │                                             │
  │  ┌───────────────────────────────────────┐  │
  │  │  sliding window buffer (up to 16MB)   │  │
  │  │  [████████████░░░░░░░░░░░░░░░░░░░░░]  │  │
  │  │   ▲ decoder_pos    ▲ bytes_received   │  │
  │  └───────────────────────────────────────┘  │
  │                                             │
  │  has_moov: bool      probe_mode: bool       │
  │  total_size: u64     duration: f64          │
  └─────────────────────────────────────────────┘
        │                               │
        ▼                               ▼
  should_throttle()              Read + Seek impl
  backpressure gate              returns zeros in
  if buffer too far              probe_mode
  ahead of decoder

The Probe Mode Problem

When symphonia first encounters an MP4 stream, it probes the file to find the moov atom and read track metadata. During probing, it reads sequentially through the file, skipping over the massive mdat (media data) atom by seeking past it.

The problem: the probe reads update decoder_pos, but the decoder hasn't actually decoded any media yet. If the download thread has received 30–300MB of data by the time probing finishes, the throttle check (bytes_received > decoder_pos + 16MB) sees the buffer as "caught up" and doesn't throttle. But then when the real decode starts and seeks back to the beginning of mdat, decoder_pos is still at the probe position — creating a permanent throttle deadlock.

Backpressure: should_throttle()

The throttle logic prevents unbounded memory growth while ensuring the decoder always has data available:

fn should_throttle(&self) -> bool {
    if self.decoder_pos > 0 {
        // Decoder is actively reading: stay 16MB ahead
        self.bytes_received > self.decoder_pos + 16 * 1024 * 1024
    } else {
        // Decoder hasn't started: buffer up to 16MB after moov found
        self.has_moov && self.buffer_size() > 16 * 1024 * 1024
    }
}

Deferred Tail Probe

Some Internet Archive files have moov at the end. Symphonia needs the moov atom before it can decode anything. Naively fetching the last 8MB at the start of every stream wastes bandwidth and triggers CDN throttling. Instead, the deferred tail probe only launches after >8MB of body data has been received without finding a moov atom. This correctly handles both moov-at-start (no tail probe needed) and moov-at-end (tail probe after 8MB).

CDN Failover

Archive.org's CDN nodes (dn*.us.archive.org) issue time-limited authentication tokens in their redirect URLs. When the streaming pipeline needs to restart a download at a byte offset (after discovering moov position), Range requests to the original CDN URL fail with 401 Unauthorized because the token has expired.

Fix: open_range_connection() routes Range requests through the original archive.org URL, which issues a fresh 302 redirect to a (possibly different) CDN node with a new token. The function follows redirect chains (301/302/303/307/308) automatically.

Seek Restart

After probe discovers the moov atom position and duration, the download restarts from an estimated byte offset via a Range request. The estimation uses linear interpolation:

let byte_offset = (seek_seconds / total_duration) * total_file_size;

This matches symphonia's SeekMode::Coarse behavior and is accurate enough for streaming playback. The prebuffer gate ensures at least 2MB (MIN_PREBUFFER) of data is available before the decoder attempts to seek.

PSP Architecture: In-Memory Streaming

The PSP pipeline is fundamentally different. With only 32MB of RAM, a 333MHz CPU, and no std::sync atomics, the desktop's Arc<Mutex> sliding window isn't viable. Instead, the PSP uses sequential in-memory streaming with hardware audio decode.

PSP Streaming Pipeline

  I/O Thread (512KB stack)
  ───────────────────────
  HTTP(S) GET file.mp4
        │
        ▼
  Buffer moov atom (~1-3MB)
        │
        ▼
  Parse with demux_lite::Mp4Lite
  (lightweight no-std MP4 parser)
        │
        ▼
  Extract interleaved samples
  from mdat in file-offset order
        │
        ├── Video H.264 NALUs → skip
        │   (ME hardware not wired for streaming)
        │
        └── Audio AAC frames
            │
            ▼
      sceAudiocodecDecode
      (hardware AAC decoder)
            │
            ▼
      AudioChannel::output_blocking
      (PSP audio hardware)
            │
            ▼
      Backpressure: output_blocking
      blocks when audio queue full,
      naturally throttling download
      to real-time playback speed

The No-Std Demuxer

The desktop uses symphonia for demuxing, but symphonia requires std and uses std::sync::Once (which doesn't work on PSP's threading model). The oasis-video crate provides demux_lite::Mp4Lite, a lightweight MP4 parser that operates on byte slices with no allocator, no std, and no synchronization primitives. It parses the moov atom into a flat array of sample entries (offset, size, track ID, timestamp) and hands them back for sequential extraction from the mdat stream.

Hardware AAC Decode

The PSP has a dedicated audio codec DSP accessible via sceAudiocodec* syscalls. The I/O thread feeds AAC frames to the hardware decoder, which produces PCM samples that go directly to the audio output channel. The key insight is that AudioChannel::output_blocking blocks when the audio hardware's command queue is full — which happens when playback is keeping up with decode. This blocking naturally throttles the download to real-time speed, acting as the PSP's equivalent of the desktop's should_throttle().

Weak Import Stubs

The sceAudiocodec and sceVideocodec firmware modules aren't loaded at boot. They're loaded lazily via sceUtilityLoadModule() at runtime. Their import stubs in the PRX must use the weak import flag (0x0008) so the PSP kernel doesn't fail module loading when the codec libraries aren't yet available.

sceVideocodec was initially declared with flag 0x4001 (strong import), which broke module loading on real hardware. Changing to 0x4009 (weak) fixed it — and this discovery was critical for the entire video pipeline. See Entry 04 for details.

The Five-Channel Result

All five TV Guide channels work on both desktop and PSP:

The same oasis-video crate handles both paths via feature flags: h264 + video-decode for desktop, no-std-demux for PSP. The TV Guide app in oasis-core is platform-agnostic — it calls the same scheduling and playback API regardless of backend.