NMS Cockpit Video

Vulkan Injection

Hooks directly into the game's Vulkan render pipeline. Renders a textured quad in 3D space, positioned relative to the cockpit camera. Works in fullscreen and borderless modes.

VR Support

Intercepts OpenVR's IVRCompositor::Submit to render per-eye stereo views. The video screen appears as a physical object in VR space with proper depth and perspective.

YouTube Playback

Load any YouTube URL directly. The daemon uses yt-dlp to extract stream URLs and decodes with ffmpeg. Hardware-accelerated via D3D11VA with automatic software fallback.

Multiplayer Sync

P2P synchronization via laminar. One player hosts, others auto-sync playback position with drift correction. Clock offset estimated via NTP-lite ping/pong exchanges.

Components

Four components work together, each running as a separate process.

Launcher

The launcher orchestrates the entire workflow:

Starts the daemon process
Launches NMS with --disable-eac (required for injection)
Waits for the game process to appear
Injects the DLL into the game
Monitors all processes and shuts down the daemon on exit

Daemon

The daemon handles all heavy processing outside the game:

Video decoding — ffmpeg-next with D3D11VA hardware acceleration, software fallback
Frame transport — Writes decoded RGBA frames to triple-buffered shared memory via seqlock
Audio playback — cpal with ringbuf for buffered output
YouTube support — yt-dlp integration for URL extraction
IPC server — Accepts commands from injector and overlay
Multiplayer sync — P2P networking via laminar (optional)

Memory budget: < 30 MB (excluding video buffer)

Injector (DLL)

The injector is a Vulkan rendering hook DLL loaded into the NMS process:

Vulkan hooks — Intercepts vkCreateDevice, vkCreateSwapchainKHR, vkQueuePresentKHR
VR hooks — Swaps the IVRCompositor::Submit vtable entry for stereo rendering
Renderer — Full Vulkan pipeline: descriptor sets, render pass, textured quad geometry
Camera reading — Reads view matrix from cGcCameraManager at known memory offsets
Keyboard input — F5-F9 hotkeys for toggling, play/pause, seeking, URL loading
Shaders — WGSL compiled to SPIR-V at build time via naga (no Vulkan SDK required)

Memory budget: < 5 MB. Requires nightly Rust.

Overlay (Legacy)

A simpler alternative for desktop-only use:

Separate transparent window rendered on top of NMS
egui for interactive controls (play/pause, seek bar, URL input)
wgpu for GPU-accelerated rendering
Reads frames from shared memory (same as injector)
Click-through mode (WS_EX_TRANSPARENT) with F9 toggle for interaction

Requires borderless or windowed mode. Does not work in VR.

Rendering Pipeline

Desktop Path

vkQueuePresentKHR hook

Read camera view matrix from memory
Poll shared memory for new frame
Compute Model-View-Projection matrix
Transition swapchain image layout
Begin render pass (LOAD existing content)
Draw textured quad with WGSL shaders
Transition swapchain image back
Submit command buffer to queue

VR Path

IVRCompositor::Submit hook

Intercept VRVulkanTextureData_t per eye
Extract VkImage from texture data
Create temporary framebuffer for eye
Begin render pass (LOAD existing content)
Draw textured quad with stereo projection
End render pass and clean up
Pass modified frame to compositor

WGSL Shader (Fragment)

@group(0) @binding(0) var tex: texture_2d<f32>;
@group(0) @binding(1) var samp: sampler;

@fragment
fn fs_main(@location(0) uv: vec2<f32>) -> @location(0) vec4<f32> {
    return textureSample(tex, samp, uv);
}

Vulkan Hook Points

Hook	Method	Purpose
`vkCreateInstance`	retour static_detour	Capture VkInstance for ash loader initialization
`vkCreateDevice`	retour static_detour	Capture VkDevice, VkPhysicalDevice, and queue family index
`vkCreateSwapchainKHR`	retour + ICD RawDetour	Track swapchain format, extent, and image handles
`vkQueuePresentKHR`	retour + ICD RawDetour	Render textured quad before desktop frame present
`IVRCompositor::Submit`	vtable swap	Render quad per VR eye before compositor submission

Usage

Quick Start

# Launch everything (daemon + NMS + injection)
nms-video-launcher.exe

# Or start components individually:
nms-video-daemon.exe --source "https://youtube.com/watch?v=..."
nms-video-daemon.exe --source "/path/to/video.mp4"

Injector Keyboard Shortcuts

Key	Action
`F5`	Toggle video overlay on/off
`F6`	Play / Pause
`F7`	Seek backward 10 seconds
`F8`	Seek forward 10 seconds
`F9`	Load URL from clipboard

Overlay Keyboard Shortcuts

Key	Action
`F9`	Toggle interactive mode (click-through ↔ interactive)
`Space`	Play / Pause
`Left / Right`	Seek backward / forward

Multiplayer

P2P sync requires EAC to be disabled on all players. One player hosts, others auto-discover via UDP broadcast on port 7332. Playback state syncs every 500ms with drift correction.

Reverse Engineering

Camera data is read from NMS memory to position the video quad in 3D space.

cGcCameraManager Memory Offsets

Singleton pointer: NMS.exe + 0x56666B0

cGcCameraManager layout:
  +0x118  Camera mode     (u32, cockpit = 0x10)
  +0x130  View matrix     (4x4 f32, row-major)
  +0x1D0  FoV             (f32, degrees)
  +0x1D4  Aspect ratio    (f32)

View Matrix Format (row-major):
  [Rx Ry Rz 0]   Right vector
  [Ux Uy Uz 0]   Up vector
  [Fx Fy Fz 0]   Forward vector
  [Px Py Pz 1]   Position

Note: Memory offsets are version-specific and may shift with NMS updates. The mem-scanner tool can re-scan for updated offsets using RTTI patterns.

Pattern Scanning Strategies

RTTI-Based (most robust, cross-version) — Scan for RTTI type descriptors like cGcCameraManager class name strings, then follow vtable references to find the singleton.
Global Pointer (fastest, version-specific) — Direct read from known offset. Fastest approach but breaks on every game update.
Code Signature (medium robustness) — Scan for instruction patterns that reference the camera manager. More resilient to minor updates.

The mem-scanner tool (in tools/mem-scanner/) scanned 2.87 GB of NMS memory in seconds to discover these offsets.

Camera Behaviour Types

NMS has 25 camera behaviour classes. The injector checks for cockpit mode (0x10) before rendering:

Mode	Value	FoV	Aspect
Cockpit (ModelView)	`0x10`	75°	1.5
On-Foot (FirstPerson)	`0x40`	70°	1.0
Photo Mode	`0x04`	Variable	Variable

Building from Source

Prerequisites

Rust (nightly for injector) ffmpeg 7.0+ dev libraries Vulkan SDK (optional) yt-dlp (optional, for YouTube)

# Clone
git clone https://github.com/AndrewAltimit/game-mods.git
cd game-mods

# Build all (release)
cargo build --release

# Build specific component
cargo build --release -p nms-video-daemon
cargo build --release -p nms-video-overlay

# Run tests
cargo test

# Format + lint
cargo fmt --all -- --check
cargo clippy --all-targets -- -D warnings

Limitations

PC only (Windows primary, Linux via LD_PRELOAD)
Singleplayer or P2P co-op only (EAC must be disabled)
Overlay mode requires borderless/windowed
NMS updates may shift memory offsets (re-scan with mem-scanner)