TLS 1.3 on a 2004 Handheld - OASIS_OS Developer's Journal

The Constraint

The PlayStation Portable's built-in SSL stack (sceSsl) uses root certificates that expired in 2008 and speaks SSL 3.0 — a protocol version that modern servers reject outright. Any PSP homebrew that wants to download from HTTPS servers (which is effectively all of the modern internet) faces a wall: the firmware's crypto is dead.

Existing PSP homebrew simply gave up on HTTPS. We didn't. The goal was clear: implement TLS 1.3 in pure Rust, running on the PSP's 333MHz MIPS Allegrex CPU, with no C dependencies, no assembly, and no certificate validation (the PSP has no trusted root store, and we're connecting to known hosts).

The Stack

PSP TLS 1.3 Network Stack

┌──────────────────────────────────────────────────────────────┐
│  Application Layer                                          │
│  HTTP client · TV Guide streaming · Internet Radio           │
├──────────────────────────────────────────────────────────────┤
│  TLS 1.3 — embedded-tls (pure Rust, no C/asm)              │
│  UnsecureProvider (no cert validation)                       │
│  alloc feature REQUIRED (RSA signature scheme advertisement) │
├──────────────────────────────────────────────────────────────┤
│  embedded-io adapters                                       │
│  Read + Write traits wrapping raw PSP sockets                │
├──────────────────────────────────────────────────────────────┤
│  Raw TCP — sceNetInet* syscalls                              │
│  sceNetInetSocket → sceNetInetConnect → sceNetInetSend/Recv  │
├──────────────────────────────────────────────────────────────┤
│  DNS — psp::net::resolve_hostname                            │
│  to_ne_bytes() — network byte order fix for little-endian     │
├──────────────────────────────────────────────────────────────┤
│  PSP WiFi Hardware — sceWlan* · sceNetAdhoc*                  │
└──────────────────────────────────────────────────────────────┘

From WiFi radio to TLS 1.3 — every layer is Rust

The implementation uses embedded-tls, a pure Rust TLS 1.3 library designed for embedded systems. It operates over the embedded_io::Read + Write traits, which we implement as thin wrappers around PSP's raw sceNetInet* syscalls. No std::net, no libc, no OpenSSL.

Discovery 1: The Privileged Instruction Trap

Discovery

mfc0 $9 (read COP0 Count register) is privileged on PSP Allegrex. On standard MIPS R4000, this instruction is available in user mode. On the PSP, it causes an immediate CPU exception — crashing the application with no backtrace.

TLS requires cryptographic randomness. The embedded-tls library uses a seed-based RNG, and the natural choice on MIPS is to read the CPU's cycle counter register ($9 in COP0). This works on every other MIPS platform.

On the PSP Allegrex, Sony restricted COP0 access to kernel mode. User-mode code that executes mfc0 $9, $0 triggers a privileged instruction exception. The crash manifests as a silent hang or address-error exception with no useful diagnostic information.

Fix: Replace the COP0 counter read with sceKernelGetSystemTimeLow(), a user-mode syscall that returns a microsecond-resolution timer. Not cryptographically random, but sufficient for seeding a PRNG when connecting to known hosts with certificate validation disabled.

// WRONG: crashes on PSP Allegrex (privileged instruction)
// let seed: u32;
// unsafe { core::arch::asm!("mfc0 {}, $9", out(reg) seed); }

// CORRECT: user-mode timer syscall
let seed = unsafe { psp::sys::sceKernelGetSystemTimeLow() } as u64;

Discovery 2: The RSA Handshake Failure

After fixing the RNG seed, TLS handshakes with archive.org consistently failed with HandshakeFailure. The same code worked fine with servers using ECDSA certificates. The failure was specific to RSA-signed certificates.

Discovery

Without the alloc feature enabled in embedded-tls, the library cannot advertise RSA signature schemes during the TLS ClientHello. The server sees a client that only supports ECDSA, finds no matching signature algorithm for its RSA certificate, and rejects the handshake.

The alloc feature is required because RSA signature verification uses heap-allocated buffers for the modular exponentiation. Without it, the RSA code paths are compiled out entirely — including the signature scheme advertisement in ClientHello. The server never knows the client could handle RSA.

# Cargo.toml — the alloc feature is NOT optional for RSA servers
[dependencies]
embedded-tls = { version = "0.17", default-features = false, features = ["alloc"] }

This is a particularly subtle failure because the TLS handshake technically completes the version negotiation and cipher suite selection — it fails during signature algorithm negotiation. The error message gives no indication that it's a missing feature flag rather than a protocol error.

Discovery 3: DNS Endianness

With TLS working, HTTP(S) connections to archive.org produced correct responses — but connections to CDN nodes (where archive.org redirects for actual file downloads) failed silently. The resolved IP addresses were wrong.

Discovery

sceNetResolverStartNtoA stores the resolved IP in in_addr format (network byte order = big-endian). On the PSP's little-endian MIPS, reading this as a u32 byte-swaps it. Using to_be_bytes() to extract octets double-swaps, producing a completely wrong IP address.

DNS Resolution Byte Order Bug

Server IP (correct):      207.241.224.2  → bytes: [CF, F1, E0, 02]

sceNetResolverStartNtoA stores in_addr (network order, big-endian):
  Memory: [CF, F1, E0, 02]

Reading as u32 on little-endian MIPS:
  u32 value = 0x02E0_F1CF  (byte-swapped by CPU)

to_be_bytes() on this u32 (WRONG — double swap):
  [02, E0, F1, CF] → 2.224.241.207

to_ne_bytes() on this u32 (CORRECT — preserves memory layout):
  [CF, F1, E0, 02] → 207.241.224.2

to_be_bytes double-swaps on little-endian — use to_ne_bytes

The fix is a one-line change: to_be_bytes() → to_ne_bytes(). But finding it required tracing network traffic on real hardware to notice that DNS responses were coming back with reversed IP octets. This bug was contributed back as PR #21 in the rust-psp SDK.

HTTP→HTTPS Redirect Handling

Archive.org's CDN randomly assigns servers per session. Some CDN nodes (ia*.us.archive.org) serve HTTP directly. Others (dn*.us.archive.org) respond with a 301 redirect to HTTPS. This creates a uniquely PSP-specific problem:

Archive.org CDN Flow on PSP

Request: GET http://archive.org/download/file.mp4

Case A: HTTP-friendly CDN node
  302 → http://ia601234.us.archive.org/file.mp4
  200 OK → stream data

Case B: HTTPS-only CDN node
  302 → http://dn543210.us.archive.org/file.mp4
  301 → https://dn543210.us.archive.org/file.mp4  ← redirect loop!

PSP sceHttp default behavior:
  Auto-follows redirects → tries HTTPS → SSL 3.0 handshake fails
  Error: 0x80431079 (misdiagnosed as connection pool corruption)

Fix: sceHttpDisableRedirect(template_id)
  Detect HTTP→HTTPS redirect → switch to raw TCP + embedded-tls
  Connect to CDN host:443 → TLS 1.3 handshake → stream data

CDN randomness creates a protocol-level routing problem

Critical Fix

sceHttpDisableRedirect(template_id) is required on PSP. Without it, the HTTP library auto-follows HTTP→HTTPS redirects, which inevitably fail because the built-in SSL can't connect to modern servers. The error code 0x80431079 is misleading — it was initially misdiagnosed as connection pool corruption before redirect tracing revealed the true cause.

The Flush Requirement

One final subtlety: embedded-tls buffers outgoing data internally for record framing. After calling write_all() on the TLS connection, the data sits in an internal buffer until flush() is called. On the PSP, where the underlying transport is a raw socket with no Nagle algorithm, forgetting flush() means the TLS record is never sent and the connection hangs waiting for a response that will never come.

// WRONG: data sits in TLS buffer, never sent
tls.write_all(request.as_bytes())?;
// ... hangs forever waiting for response

// CORRECT: flush pushes the TLS record out
tls.write_all(request.as_bytes())?;
tls.flush()?;  // ← sends the actual TLS record

Performance on 333MHz MIPS

TLS 1.3 handshakes on the PSP take approximately 2–4 seconds, dominated by the key exchange computation (ECDHE with P-256). This is acceptable for the use case: establishing a streaming connection that will transfer megabytes of data. The per-record overhead after handshake is negligible — symmetric AES-GCM encryption at 333MHz handles the streaming bitrate without issue.

The I/O thread stack was increased from 256KB to 512KB to accommodate the TLS library's stack usage during key exchange. Embedded-tls allocates several large buffers on the stack during handshake, and 256KB proved insufficient for the RSA code path.

Result

The PSP can now connect to any TLS 1.3 server. The Internet Archive's TV Guide streams audio over HTTPS from CDN nodes that refuse HTTP. Internet Radio connects to stations that have migrated to HTTPS-only. The browser can load HTTPS pages. All running on a 2004 handheld that Sony expected to speak SSL 3.0 to servers that no longer exist.