3D Graphics & Rendering

3D graphics rendering transforms mathematical representations of three-dimensional scenes into two-dimensional images on screen. Modern pipelines combine geometry algorithms, parallel GPU architectures, and advanced shading to produce photorealistic or stylized visuals in real-time for games, simulations, and interactive applications. Four ideas frame the whole subject:

• A pipeline, not magic. Every frame is a fixed sequence: transform geometry, decide which pixels it covers, then shade them. Knowing the stages tells you where time goes.
• Massively parallel. A GPU runs thousands of threads at once; performance comes from feeding that parallelism evenly, not from clever serial tricks.
• Rasterize vs trace. Rasterization is fast and approximate; ray tracing is accurate and expensive. Modern engines blend both (“hybrid rendering”).
• Budget-driven. At 60 FPS you have ~16 ms per frame — rendering is a constant negotiation between visual fidelity and that fixed time budget.

The Rendering Pipeline

Overview

Rendering answers two questions for every pixel: what surface is visible here? and what color is that surface? The graphics pipeline processes geometry through a fixed sequence of stages to answer them. Vertices flow in at the top, get transformed and assembled into triangles, are converted to candidate pixels (fragments), and finally shaded and composited into the framebuffer.

flowchart TD
    A["Application Stage (CPU)<br/>scene setup, culling, draw calls"] --> B["Vertex Shader<br/>transform vertices to clip space"]
    B --> C{"Optional<br/>geometry stages"}
    C --> D["Tessellation<br/>subdivide patches"]
    C --> E["Geometry Shader<br/>emit/modify primitives"]
    D --> F["Clipping & Culling<br/>discard off-screen / backfaces"]
    E --> F
    C --> F
    F --> G["Rasterization<br/>triangles to fragments"]
    G --> H["Fragment / Pixel Shader<br/>compute color & lighting"]
    H --> I["Depth & Stencil Test"]
    I --> J["Blending"]
    J --> K["Framebuffer Output"]

The CPU-side application stage is where your engine lives: it decides what to draw, performs high-level culling, and issues draw calls. Everything below it runs on the GPU. The fixed-function stages (rasterization, depth test, blend) are configured but not programmed; the programmable stages (vertex, fragment, and optionally tessellation/geometry/compute) are where shader code runs.

Coordinate Spaces

Transformations move vertices through coordinate systems:

Vertex Processing

The vertex shader transforms each vertex:

// Basic vertex shader
#version 450

layout(location = 0) in vec3 inPosition;
layout(location = 1) in vec3 inNormal;
layout(location = 2) in vec2 inTexCoord;

layout(binding = 0) uniform Matrices {
    mat4 model;
    mat4 view;
    mat4 projection;
};

layout(location = 0) out vec3 fragPosition;
layout(location = 1) out vec3 fragNormal;
layout(location = 2) out vec2 fragTexCoord;

void main() {
    vec4 worldPos = model * vec4(inPosition, 1.0);
    fragPosition = worldPos.xyz;
    fragNormal = mat3(transpose(inverse(model))) * inNormal;
    fragTexCoord = inTexCoord;

    gl_Position = projection * view * worldPos;
}

Rasterization vs Ray Tracing

The two dominant paradigms for visibility determination take opposite approaches. Rasterization projects each triangle onto the screen and asks “which pixels does it cover?” — it iterates over geometry. Ray tracing shoots a ray through each pixel and asks “what does this ray hit?” — it iterates over pixels. Rasterization is the basis of real-time graphics because it maps perfectly onto GPU parallelism and avoids expensive scene-wide queries; ray tracing produces physically correct shadows, reflections, and global illumination but historically required offline render farms.

Hybrid rendering is the norm. Modern engines (UE5, Frostbite, RED Engine) rasterize primary visibility for speed, then fire rays only for the effects where rasterization fails — shadows, reflections, and indirect light. This pays the ray-tracing cost only where it buys the most visual quality; UE5’s Lumen and ray-traced shadows are exactly this strategy.

Lighting and Shading

Lighting Models

Phong Reflection Model:

Blinn-Phong (Optimized):

• Uses the halfway vector $\vec{H} = \operatorname{normalize}(\vec{L} + \vec{V})$
• Specular term becomes $k_s\,(\vec{N}\cdot\vec{H})^{\,n}$
• More efficient, with near-identical results

Physically Based Rendering (PBR)

Modern standard for realistic materials:

Core Parameters:

• Albedo/Base Color: Surface color without lighting
• Metallic: Metal (1.0) vs dielectric (0.0)
• Roughness: Microsurface irregularity (0.0 = smooth, 1.0 = rough)
• Normal Map: Per-pixel surface detail
• Ambient Occlusion: Soft shadowing in crevices

Cook-Torrance BRDF:

• $D$ — Normal Distribution Function (GGX / Trowbridge-Reitz)
• $F$ — Fresnel term (Schlick approximation)
• $G$ — Geometry / shadowing-masking (Smith GGX)

Global Illumination

Simulating indirect lighting:

Real-Time Techniques:

• Screen-Space GI (SSGI): Sample nearby pixels
• Voxel Cone Tracing: Voxelize scene, trace cones
• Light Probes: Precomputed irradiance at points
• Reflection Probes: Cubemap captures for reflections
• Ray Tracing: Hardware-accelerated path tracing

Unreal Engine 5 Lumen:

• Hybrid software/hardware ray tracing
• Infinite bounces for indirect light
• Dynamic, no baking required
• Screen-space fallback for efficiency

Shadow Rendering

Shadow Mapping

Standard real-time shadow technique:

1. Shadow Pass: Render depth from light’s perspective
2. Main Pass: Compare fragment depth to shadow map
3. Result: In shadow if fragment depth > shadow map depth

Common Issues and Solutions:

• Shadow Acne: Add depth bias
• Peter Panning: Reduce bias, use slope-scaled bias
• Aliasing: PCF filtering, variance shadow maps
• Resolution: Cascaded shadow maps for large scenes

Cascaded Shadow Maps (CSM)

Multiple shadow maps for different distance ranges:

Near cascade: High resolution, close to camera
Mid cascade: Medium resolution, mid-range
Far cascade: Low resolution, distant objects

Split distances based on:
- Logarithmic distribution
- Practical split scheme (PSSM)
- Custom per-game tuning

Ray-Traced Shadows

Hardware ray tracing benefits:

• Pixel-perfect accuracy
• Natural soft shadows from area lights
• No aliasing or bias issues
• Higher performance cost

Advanced Rendering Techniques

Deferred Rendering

Separate geometry from lighting:

G-Buffer Contents:

• Position (or depth for reconstruction)
• Normal
• Albedo/Diffuse
• Specular/Roughness
• Emissive (optional)

Advantages:

• Decouple geometry complexity from light count
• Efficient many-light scenarios
• Easy post-processing access to scene data

Disadvantages:

• High memory bandwidth
• Difficult transparency handling
• MSAA complications

Forward+ Rendering

Hybrid approach:

1. Depth Pre-Pass: Populate depth buffer
2. Light Culling: Tile-based light assignment
3. Shading: Forward pass with culled light lists

Benefits:

• Supports transparency naturally
• Lower memory bandwidth than deferred
• MSAA compatible
• Efficient for moderate light counts

Clustered Rendering

3D extension of Forward+:

• Divide view frustum into 3D clusters
• Assign lights to clusters (not just tiles)
• Better handling of depth discontinuities
• More uniform light distribution

Post-Processing Effects

Screen-Space Effects

Ambient Occlusion:

• SSAO (Screen-Space Ambient Occlusion)
• HBAO+ (Horizon-Based)
• GTAO (Ground Truth)

Reflections:

• SSR (Screen-Space Reflections)
• Hi-Z tracing for efficiency
• Fallback to probes for missing data

Motion Blur:

• Per-object velocity buffers
• Camera motion blur
• Temporal reconstruction

Color Grading and Tone Mapping

HDR to LDR Conversion:

// Reinhard tone mapping
vec3 toneMapReinhard(vec3 hdrColor) {
    return hdrColor / (hdrColor + vec3(1.0));
}

// ACES Filmic
vec3 toneMapACES(vec3 x) {
    float a = 2.51;
    float b = 0.03;
    float c = 2.43;
    float d = 0.59;
    float e = 0.14;
    return clamp((x*(a*x+b))/(x*(c*x+d)+e), 0.0, 1.0);
}

Color Grading:

• LUT (Look-Up Table) based
• Split toning (shadows/highlights)
• Color wheels adjustment
• Film grain and vignette

Anti-Aliasing

Techniques Comparison:

*Upscaling provides net performance gain

Temporal Anti-Aliasing (TAA)

Modern standard approach:

1. Jitter projection matrix each frame
2. Accumulate samples over time
3. Reject samples using motion vectors
4. Apply neighborhood clamping

Challenges:

• Ghosting on fast motion
• Loss of fine detail
• Requires motion vectors

GPU Architecture

Parallelism Model

GPUs execute thousands of threads simultaneously. Hardware is organized as a hierarchy of compute units, and the software thread model mirrors it:

flowchart TD
    GPU["GPU"] --> SM["Streaming Multiprocessors (SM)"]
    GPU --> L2["L2 Cache"]
    GPU --> VRAM["Video Memory (VRAM)"]
    SM --> Cores["CUDA Cores / Stream Processors"]
    SM --> SMem["Shared Memory"]
    SM --> L1["L1 Cache"]
    subgraph Thread Model
      Grid["Grid (whole dispatch)"] --> Block["Thread Block (shares memory)"]
      Block --> Warp["Warp / Wavefront (32 / 64 lock-step threads)"]
      Warp --> Thread["Thread (one lane)"]
    end

Threads within a warp execute in lock-step (SIMT). When threads in a warp take different branches, the GPU runs both paths and masks the inactive lanes — this is warp divergence, the reason branchy shaders are slow. Keeping all 32/64 lanes doing the same work is the single most important shader-performance principle.

Memory Hierarchy

Optimizing for GPU memory access:

Graphics APIs

Vulkan:

• Low-level, explicit control
• Cross-platform
• Best for engine developers

DirectX 12:

• Windows/Xbox exclusive
• Similar to Vulkan
• Better tooling ecosystem

Metal:

• Apple platforms
• Excellent iOS/macOS integration
• Swift/Objective-C friendly

WebGPU:

• Browser-based 3D
• Modern API design
• Growing adoption

Optimization Techniques

Culling

Eliminate invisible geometry:

• Frustum Culling: Outside camera view
• Occlusion Culling: Hidden behind other objects
• Backface Culling: Faces pointing away from camera
• Distance Culling: Beyond view distance
• Small Object Culling: Sub-pixel geometry

Level of Detail (LOD)

Reduce complexity with distance:

LOD 0: Full detail (0-50m)
LOD 1: 50% triangles (50-100m)
LOD 2: 25% triangles (100-200m)
LOD 3: 10% triangles (200m+)
Billboard: 2D impostor (very far)

Modern Approaches:

• Nanite (UE5): Virtualized geometry, automatic LOD
• Mesh Shaders: GPU-driven LOD selection
• Continuous LOD: Smooth transitions

Batching and Instancing

Reduce draw calls:

• Static Batching: Combine static meshes
• Dynamic Batching: Runtime combination of small meshes
• GPU Instancing: Single draw call, multiple instances
• Indirect Drawing: GPU-driven draw commands

Key Takeaways

• The pipeline is fixed. Vertices → triangles → fragments → framebuffer. Knowing which stage dominates a frame is the first step in optimizing it.
• Rasterize first, trace selectively. Rasterization wins on speed; ray tracing wins on accuracy. Hybrid pipelines rasterize visibility and trace only shadows, reflections, and GI.
• PBR is the material standard. Albedo, metallic, and roughness drive a physically grounded Cook-Torrance BRDF that behaves consistently under any lighting.
• Draw calls and overdraw cost. Batching, instancing, LODs, and culling exist to reduce CPU-side draw calls and wasted GPU shading.
• Avoid warp divergence. The GPU shades 32/64 threads in lock-step. Branchy shaders and uneven work waste lanes — keep work uniform.
• TAA + upscaling are default. Temporal accumulation (TAA) plus DLSS/FSR now deliver the best quality-per-millisecond for anti-aliasing.

See Also

• Game Development - Game engines, physics, and multiplayer systems
• Performance Optimization - GPU profiling and optimization techniques
• Unreal Engine - UE5 Nanite, Lumen, and MetaSounds
• VR/AR Development - Immersive rendering and XR techniques
• Physics Documentation - Mathematical foundations