Modernizing the rust-psp SDK: Safety Hardening for Bare Metal MIPS

Why Fork?

The original rust-psp is a pioneering project that proved Rust could target the PSP's MIPS Allegrex CPU. But it was designed for simple homebrew demos, not a production OS. OASIS_OS needed:

• Kernel mode support for PRX plugins (syscall hooking, MMIO access)
• High-level modules wrapping the 829 raw syscall bindings into safe APIs
• Fixes for bugs that only manifest on real hardware (not the PPSSPP emulator)
• Edition 2024 compatibility and modern Rust safety annotations
• RAII resource management for audio channels, files, threads, network sockets

The fork at AndrewAltimit/rust-psp now contains 37,000+ lines of Rust across 50+ source files, with 38 high-level SDK modules and 30 example programs.

The LLVM Memcpy Recursion Bug

This is a known LLVM behavior on MIPS (and some other targets), but it's particularly insidious because:

• It doesn't happen on x86, so it passes all CI tests on desktop
• On PSP, it manifests as a "jump to invalid address" crash, not a stack overflow
• The PSP has no stack guard page, so the recursion silently corrupts memory before crashing
• Optimized builds can sometimes avoid the recursion by inlining, making the bug intermittent

The fix is to implement memcpy, memset, and memmove using manual byte-by-byte loops that LLVM cannot lower back into intrinsic calls:

#[no_mangle]
pub unsafe extern "C" fn memset(dest: *mut u8, val: i32, n: usize) -> *mut u8 {
    let val = val as u8;
    let mut i = 0;
    while i < n {
        // Manual byte write — LLVM cannot lower this to memset
        core::ptr::write_volatile(dest.add(i), val);
        i += 1;
    }
    dest
}

The write_volatile is critical: it prevents LLVM from recognizing the loop as a memset pattern and "optimizing" it back into a memset call. This is a compiler barrier, not a hardware requirement.

Weak Import Stub Flags

PSP firmware modules export functions identified by 32-bit NIDs (Name IDs). When a PRX is loaded, the kernel resolves its import stubs against loaded modules. The stub declaration includes a flags field that controls resolution behavior:

PSP Import Stub Resolution

psp_extern! macro generates stub declarations:

  struct SceStubLibraryEntry {
      name: *const c_char,     // library name
      flags: u16,              // resolution flags
      num_funcs: u16,          // number of imported functions
      nid_table: *const u32,   // NID array
      stub_table: *const u32,  // stub function array
  }

Flag Bits:
  0x0001  → standard import (module must be loaded)
  0x0008  → weak import (module may not be loaded yet)
  0x4000  → library version hint

Impact:
  0x4001 (strong)  Module fails to load if library not present
  0x4009 (weak)    Module loads; stubs resolve later via sceUtilityLoadModule

The 38 High-Level Modules

The raw psp::sys module exposes 829 syscall bindings as unsafe C-ABI functions. The SDK wraps these into 38 high-level modules with safe Rust APIs:

rust-psp SDK Module Map

Threading & Sync
├── thread.rs      closure-based spawn (bypasses std TLS limitation)
├── sync.rs        SpinMutex, SpinRwLock, SpscQueue (lock-free)
└── timer.rs       FrameTimer, alarm callbacks

Graphics
├── gu_ext.rs      sprite batching, texture helpers
├── framebuffer.rs double-buffering with VRAM management
├── simd.rs        VFPU matrix/vector math (PSP's SIMD unit)
└── screenshot.rs  BMP capture from framebuffer

Audio
├── audio.rs       AudioChannel with RAII cleanup
├── audio_mixer.rs multi-channel mixing
├── audiocodec.rs  hardware AAC/MP3/ATRAC decode
└── mp3.rs         MP3 decoder with stability guards

Networking
├── net.rs         TcpStream, UdpSocket (RAII, Read + Write)
├── http.rs        HttpClient with persistent templates
└── wlan.rs        WiFi connection management

Input & UI
├── input.rs       controller with analog deadzone
├── osk.rs         on-screen keyboard integration
└── dialog.rs      system message dialogs

Storage
├── io.rs          File, ReadDir (RAII handles)
├── config.rs      binary config (RCFG format)
└── savedata.rs    encrypted save data management

System
├── power.rs       clock speed control, battery monitoring
├── time.rs        Instant, Duration, system clock
├── rtc.rs         real-time clock with timezone
├── usb.rs         USB device mode control
├── display.rs     VBlank sync, framerate control
├── cache.rs       data/instruction cache management
├── dma.rs         DMA transfer engine
└── vram_alloc.rs  VRAM allocator (Result-based, checked math)

Kernel Mode Only
├── me.rs          Media Engine boot, task submission
├── hw.rs          MMIO register access
├── hook.rs        SyscallHook (PRO-C2 stub workarounds)
├── font.rs        system TrueType font rendering
└── benchmark.rs   CPU cycle measurement

RAII Everywhere

Every resource-holding type implements Drop:

• File closes sceIo handles on drop
• AudioChannel releases the hardware channel
• TcpStream closes the socket
• JoinHandle joins the thread
• SpriteBatch flushes remaining vertices

This is especially important on the PSP, where leaked resources can't be reclaimed without a full reboot. There's no OS-level cleanup of user processes — when a homebrew exits, any un-freed resources stay allocated until power cycle.

Kernel Mode: SyscallHook

The PSP plugin PRX needs to hook firmware syscalls to intercept display buffer swaps. The SyscallHook implementation handles two different CFW stub formats:

PSP Syscall Hook Mechanisms

Method 1: sctrlHENPatchSyscall (preferred)
  Original stub:
    jr  $ra          ← return to caller
    syscall 0xNNNN   ← kernel trap with syscall number

  After patch:
    j   hook_func    ← jump to our hook
    nop              ← delay slot (cleaned)

  PRO-C2 on 6.20 quirk: delay slot contains garbage
  from original stub, not nop. Must read stub bytes
  and extract syscall number before patching.

Method 2: Inline Hook (fallback)
  Patch first instruction of target function:
    j   trampoline   ← jump to our code
    nop              ← delay slot

  Trampoline:
    [save registers]
    jal hook_func    ← call our hook
    [restore registers]
    [original instruction]
    j   target+8     ← resume original function

Edition 2024 Modernization

The fork updates the entire codebase to Rust edition 2024:

• #[no_mangle] → #[unsafe(no_mangle)] (new safety annotation)
• Workspace-level lints with unsafe_op_in_unsafe_fn = "warn"
• Removed 4 stabilized nightly features
• paste::paste re-exported for macro resolution compatibility
• Thread-unsafe panic counter replaced with AtomicUsize
• Allocator overflow checks via checked_add
• GU matrix bounds checking (previously unchecked array index)
• Screenshot BMP serialization rewritten from transmute to safe field-by-field writes

Testing on Real Hardware vs. PPSSPP

The CI pipeline includes PPSSPP headless testing for basic smoke tests, but the "real" test suite is manual execution on hardware. This is a fundamental constraint of bare-metal development: the emulator is a convenience tool, not a source of truth.

By the Numbers