Diffusion Model Outputs - Formats, Export, and Provenance

Diffusion and flow-matching models now generate images, video, audio, 3D assets, and, experimentally, text. This page is a reference for what each modality produces, which file formats and encoder settings suit which destination, how to get from a model’s raw output to a deliverable, and how to record provenance so outputs are reproducible and properly labelled as AI-generated. It assumes the basics from Stable Diffusion Fundamentals.

One Mechanism, Many Modalities

Every modality below uses the same recipe: start from noise in a latent space, denoise it step by step under some conditioning (text, image, audio), then decode the latent into the target medium. Only the latent’s shape and the decoder change.

flowchart LR
    N["Gaussian noise<br/>(latent tensor)"] --> D["Denoiser<br/>U-Net or transformer,<br/>N steps"]
    C["Conditioning<br/>text / image / audio"] -.-> D
    D --> Z["Clean latent"]
    Z --> I["Image VAE decoder"] --> IO["PNG / WebP / AVIF / EXR"]
    Z --> V["Video VAE decoder<br/>(spatio-temporal)"] --> VO["MP4 / WebM / ProRes"]
    Z --> A["Audio decoder<br/>or vocoder"] --> AO["WAV / FLAC / Opus"]
    Z --> G["3D decoder<br/>(mesh / splats / voxels)"] --> GO["GLB / USDZ / PLY"]

Mainstream language models (GPT, Claude, Gemini) are autoregressive, not diffusion-based. Text diffusion is covered at the end as an emerging line of work.

Images

Images are the most mature output and the one where format choice has the most visible consequences. Save a lossless master first and derive everything else from it.

Format Comparison

Practical defaults:

• PNG is the working and archival format. It is the only common format in which ComfyUI and Automatic1111 embed generation metadata by default. Use 16-bit when heavy grading follows. The PNG Third Edition (W3C Recommendation, June 2025) formally adds APNG animation, the cICP chunk for HDR, and eXIf metadata.
• JPEG at quality 80-90 with progressive encoding for sharing. Avoid repeated re-saves, because generation loss compounds.
• WebP at quality ~85-90 for web pages that need transparency or animation.
• AVIF where file size matters most. Encoding is slower, so derive it offline.
• JPEG XL has strong technical properties and Safari support, and Chromium re-added a decoder (behind a flag) in early 2026. Until it is on by default everywhere, serve it only with fallbacks.
• EXR only for compositing pipelines that need float precision or multiple named channels.

Note that model outputs are 8-bit sRGB in practice: the VAE decodes to values clamped to [0, 1] and quantized. Saving 16-bit PNG does not recover lost precision, but it prevents banding from compounding during later edits.

flowchart TD
    Q1{"Will it be edited<br/>or composited?"} -->|Yes| Q2{"VFX / multi-pass?"}
    Q2 -->|Yes| EXR["OpenEXR, 32-bit float"]
    Q2 -->|No| PNG16["PNG 16-bit or TIFF"]
    Q1 -->|No| Q3{"Destination"}
    Q3 -->|Web page| AV["AVIF or WebP,<br/>JPEG fallback"]
    Q3 -->|Social / chat| JPG["JPEG q85 at<br/>platform size"]
    Q3 -->|Print| TIF["TIFF or PNG at 300 DPI,<br/>embedded ICC profile"]
    Q3 -->|Game engine| GPU["PNG source, engine<br/>compresses to BC7 / ASTC"]

Resolution and Upscaling

Generate at the model’s native resolution (about 1 megapixel for SDXL, SD3.5, and FLUX.1, in any supported aspect ratio), then resize for the target. Generating directly at an unusual small size degrades composition, and generating far above native size causes duplicated subjects.

To reach print or 4K sizes:

1. Generate at native resolution.
2. Upscale 2-4x with a model upscaler (Real-ESRGAN, 4x-UltraSharp, or a SwinIR/DAT-family model).
3. Optionally run a low-denoise (0.2-0.45) img2img pass, tiled with a tile ControlNet or a tiled-diffusion node, to add real detail instead of enlarged pixels.

Enable tiled VAE decode for large images; VAE decoding is often the step that runs out of memory. See Advanced Techniques for hires-fix and tiled-upscale workflows.

Platform specifications change often; check the platform’s current guidance before a large export.

Text Inside Images

Legible typography depends mostly on the text encoder. CLIP-only models (SD 1.5, SDXL) cannot spell reliably; models with a T5 or LLM text encoder can.

When direct generation is not reliable enough:

1. ControlNet from rendered type. Render the words in a real font, extract Canny or lineart, and condition on it so the model styles exact letterforms.
2. Inpaint the text region. Generate the scene, mask the area, and inpaint only the lettering, possibly with a stronger model than the one that made the scene.
3. Composite in post. For logos, legal text, or anything that must be exactly right, set real type in an editor.

Describing the physical medium (“neon sign reading OPEN on a brick wall, night photograph”) works better than naming the words alone, because the model has seen that context. For print deliverables such as book covers, generate the art at print resolution and add title typography in a layout tool that exports PDF/X with bleed and CMYK conversion.

Multi-Pass and Layered Outputs

Running estimators alongside generation yields companion data for downstream tools:

Keep derived maps at the same dimensions as the color image, and mark color-space correctly: albedo is sRGB, while normal, roughness, and depth are linear data.

Video

Video models denoise a spatio-temporal latent, so frames are coherent by construction rather than generated independently. The field changed quickly in 2024-2025. Short-clip add-ons such as AnimateDiff and Stable Video Diffusion have largely given way to dedicated text/image-to-video transformers, and the leading hosted models now generate synchronized audio as well.

Models

Open video models are VRAM-hungry. Most local workflows depend on FP8 or GGUF quantization, block swapping or offloading, and tiled VAE decoding. Clip length is typically bounded by training length (roughly 5 seconds), so longer pieces are built by extending from the last frame or by editing shots together.

From Model Output to Deliverable

flowchart LR
    G["Generate<br/>16-24 fps frames"] --> U["Upscale<br/>(optional, per frame<br/>or video upscaler)"]
    U --> F["Interpolate<br/>RIFE / FILM to 24-60 fps"]
    F --> GR["Grade, stabilize,<br/>add grain"]
    GR --> AU["Mux audio<br/>(generated or scored)"]
    AU --> E["Encode per platform"]
    G -.->|lossless intermediate| M[("PNG sequence<br/>or ProRes master")]

• Keep a lossless or mezzanine master (PNG sequence, ProRes 422 HQ/4444, or DNxHR) and encode delivery files from it.
• Frame interpolation (RIFE for speed, FILM for large motion) raises low native frame rates. Interpolating generated motion can produce warping on fast action, so check the result.
• Encode with constant quality (CRF/CQ) rather than a fixed bitrate for archival and web, and let platforms re-encode uploads.

Codecs and Containers

A typical high-compatibility encode with ffmpeg:

ffmpeg -framerate 24 -i frames/%05d.png -i audio.wav \
  -c:v libx264 -preset slow -crf 18 -pix_fmt yuv420p \
  -c:a aac -b:a 192k -movflags +faststart -shortest out.mp4

-pix_fmt yuv420p is required for playback in many browsers and phones, and +faststart moves the index to the front so web playback can begin before the download completes.

Audio

Audio generators denoise a latent from a waveform autoencoder or a mel-spectrogram, then decode it to samples. Not every modern speech model is diffusion-based: many TTS systems are autoregressive codec models, while others (F5-TTS, for example) use flow matching.

Export Settings

Normalize loudness, not peaks. Common targets are about -14 LUFS integrated for YouTube and Spotify and -16 LUFS for Apple Music and podcasts, with true peak at or below -1 dBTP. Generated audio often has a sample rate below 48 kHz (many models work at 44.1 kHz or lower), so resample with a high-quality resampler before muxing with video. For game loops, generate longer than needed and cut at a zero crossing, with a short crossfade.

3D

3D generation matured quickly. Early single-image reconstructors (TripoSR, 2024) produced rough vertex-colored meshes. Current open models generate clean geometry with full PBR materials and export GLB directly.

For image-to-3D, generate the input with a single centered subject, a plain background, and even lighting (“game asset, neutral lighting, white background”). Then remove the background before reconstruction.

flowchart LR
    R["Reference image<br/>FLUX / SDXL"] --> BG["Background<br/>removal"]
    BG --> M["Image-to-3D<br/>TRELLIS.2 / Hunyuan3D"]
    M --> C["Cleanup in Blender:<br/>retopology, decimation,<br/>UV / texture fixes"]
    C --> E{"Target"}
    E -->|Web| GLB["GLB + Draco / Meshopt,<br/>KTX2 textures"]
    E -->|Unity / Unreal| FBX["FBX or GLB"]
    E -->|Apple AR| USDZ["USDZ"]
    E -->|3D print| STL["STL / 3MF, watertight"]

3D Formats

Generated meshes are usually dense, triangulated, and unevenly UV-mapped. Plan on decimation or retopology and texture baking before using them as real-time game assets.

Provenance and AI Disclosure

Metadata answers two different questions: how was this made (reproducibility) and was this made by AI (disclosure). Treat both as part of the output.

Reproducibility Metadata

Record at minimum the prompt and negative prompt; the model, LoRAs, and VAE with hashes; sampler, scheduler, steps, CFG or guidance, seed, and resolution; and the tool and version, plus the full workflow graph.

Dragging a ComfyUI PNG back into ComfyUI restores its exact graph, so keep the original PNG even when delivering other formats. Pipelines that automate this are covered in Production Pipelines.

Disclosure: Content Credentials and Watermarks

• C2PA Content Credentials attach a cryptographically signed manifest recording that the content was generated, by what tool, and any subsequent edits. Major image tools, cameras, and several social platforms read or write them. The IPTC Digital Source Type value trainedAlgorithmicMedia is the standard metadata term for fully AI-generated media.
• Invisible watermarks embed a signal in the pixels or audio itself (for example Google’s SynthID), so they can survive metadata stripping and some re-encoding.
• Regulation. The EU AI Act’s Article 50 transparency obligations, applicable from 2 August 2026, require providers of generative AI systems to mark synthetic output in a machine-readable, detectable way. Deployers must also disclose deepfakes. The Commission’s guidance points toward layered approaches (signed metadata plus watermarking) rather than any single technique. If you distribute generated media in the EU, check the current requirements and implementation deadlines for your role.

Metadata alone is fragile, because most platforms strip it on upload. Combining a signed manifest with a watermark, and keeping your own sidecar records, is the robust approach.

Post-Processing

A light finishing pass fixes common generation artifacts without overworking them:

• Images: small contrast and vibrance adjustments, sharpening matched to output size, and a denoise pass only if high CFG introduced grain. Check hands, text, and symmetric details at 100% zoom.
• Video: stabilize jitter, interpolate frames, grade to a consistent look (LUT), add light grain to hide banding, then mix audio with headroom.
• Audio: EQ, gentle compression, limiting, then loudness normalization to the target. Keep a 24-bit master.

Live Previews and Streaming

Because sampling is iterative, intermediate latents can be decoded cheaply for previews. A tiny approximate decoder (TAESD and its SDXL/SD3/FLUX variants) turns a latent into a low-resolution image in milliseconds. ComfyUI streams these previews as binary WebSocket frames (see Production Pipelines). Few-step models (LCM, Turbo, Lightning, and distilled FLUX variants) make near-interactive generation possible; see Advanced Techniques.

Text Diffusion (Experimental)

Diffusion language models (dLLMs) start from a fully masked or noised token sequence and refine all positions in parallel over a number of steps, rather than emitting one token at a time. Their appeal is throughput and the ability to revise earlier tokens.

Output handling is identical to any LLM: tokens become plain text, Markdown, JSON, or code. Nothing about output formats is diffusion-specific here, and autoregressive models remain the default for most text work.

Quick Reference

The durable strategy is a format-agnostic pipeline: keep one high-quality master with its metadata, derive target formats on demand, and let new formats (JPEG XL on the web, glTF splats) slot in without regenerating anything.

See Also

• Stable Diffusion Fundamentals - The denoising process shared by every modality
• Production Pipelines - Automating generation, metadata, and derivative exports
• ComfyUI Guide - Visual workflow creation
• Advanced Techniques - Upscaling, few-step models, and multi-stage workflows
• Inpainting and Editing - Region edits, including text repair
• ControlNet - Structural control, including rendered-text conditioning
• Base Models Comparison - Which model to use for which output
• AI/ML Documentation Hub - Complete AI/ML documentation index