COOKBOOK.md

Cosmos Cookbook

Start from what you want to achieve. Every recipe here leverages Cosmos to help you reach an objective.

Jump to: Inference · Simulation · Post-training · Distillation · Quantization · Evaluation

Legend: industry and GPU-budget codes

Code	Industry	Code	GPU budget
ANY	any industry	S	1 GPU, up to 96 GB
AV	autonomous vehicles	M	1 GPU over 96 GB, or 2–8 GPUs
ROB	robotics	L	9–64 GPUs
INFRA	smart infrastructure	XL	more than 64 GPUs
MFG	manufacturing and industrial sites
AG	agriculture
MED	healthcare

Inference

Recipe	Industry	GPU budget
Generation Text-to-image, text-to-video, image-to-video, video-to-video, with optional audio	ANY	S M
Visual Reasoning Captioning, VQA, grounding, temporal localization, physical plausibility	ANY INFRA	S M
Action Policy Forward dynamics, inverse dynamics, policy	AV ROB	S
Transfer Spatial controls: edge, blur, depth, segmentation, world-scenario map, multi-control	ANY AV	S M
NIM deployment Choose a run time (generation/reasoning), run accelerated inference from one container · support matrix	ANY	S M

Simulation 

Recipe	Industry	GPU budget
Multi-view multi-sensor driving SDG Fine-tune Cosmos on driving data for multi-view + LiDAR joint generation · Try example checkpoint	AV	L
Autoregressive driving simulation Build autoregressive multi-view driving simulator · Try example checkpoint	AV	M
Autoregressive bimanual simulation Build interactive neural simulator for robot policy· Try example checkpoint	ROB	M
Autoregressive sim-to-real transfer Build autoregressive sim-to-real transfer model · Try example checkpoint	ROB	M
Surgical simulation Coming soon	MED

Post-training

Recipe	Industry	GPU budget
Generation Post-train with your captioned video: full SFT or LoRA	ANY	M
Reasoning Alignment SFT and physical-plausibility SFT	ANY	M
Action policy Post-train a policy, then evaluate it closed-loop: DROID, LIBERO, RoboCasa	ROB	L XL
Forward dynamics Post-train a forward-dynamics model on DROID	ROB	M
Worker safety Coming soon · Flag hazards in industrial video	MFG	S
Action Policy (Light) Coming soon · Post-train a LIBERO-10 manipulation policy on one GPU, evaluate it closed-loop	ROB	M

Distillation

Recipe	Industry	GPU budget
4-step students 4-step distillation guide, text-to-image, image-to-video	ANY	L

Quantization

Recipe	Industry	GPU budget
Quantization Static FP8 export and serving: Nano/Super video, Super base and 4-step text-to-image/image-to-video	ANY	S M

Evaluation

Benchmarks	Industry	GPU budget
Generation PAI-Bench-G video · PAI-Bench-C controlled video · PhysicsIQ physics · UniGenBench images · RBench robotics	ANY ROB	M
Reasoning TAR & VANTAGE with VLMEvalKit, on any OpenAI-compatible endpoint	INFRA	S

---