Run Sim-Auto in the cloud

SimForge Factory runs the Cosmos3-Nano-Sim-Auto video generator on SimForge's GPUs. It does two kinds of work:

• Scenario clips. You name scenarios and a look (or your own prompt). SimForge renders each scenario, draws the layout videos the generator follows, generates the clip, checks it against the render and labels it. You get checked, labeled camera clips; a clip that fails the check is made again on a new seed, free.
• Raw generations. You bring the generator's inputs yourself: a control (layout) video per camera, a prompt per camera, and in simulator mode a first frame per camera. SimForge runs exactly the request a local Sim-Auto server would get, and hands back what it made. No scenario, no labels, no check.

Every scenario clip is 20 seconds, 600 frames at 30 fps, at 832x480: the generator makes 601 frames and the last is dropped. A scenario shorter than 20 s is simulated on to 20 s, and a longer one gives its first 20 s; the quote says so.

Make scenario clips#

simforge factory looks
simforge factory clip create --scenario SCENARIO_ID --look fog-dawn --look snow --dry-run
simforge factory clip create --scenario SCENARIO_ID --look fog-dawn --seed 7 --seed 8 --wait
simforge factory clip create --dataset DATASET_ID --prompt "Heavy rain at night, wet asphalt reflecting street lights." --yes

simforge factory looks lists the looks; an experimental one may not apply well. Name scenarios with --scenario (repeatable) or every scenario of a dataset with --dataset, and looks with --look (repeatable) or your own conditions with --prompt. --camera-prompt CAMERA=TEXT replaces one camera's whole prompt, and --camera (repeatable) makes fewer cameras of the 4-camera rig, priced on those cameras. --seed (repeatable) picks the seeds, or --seeds-per-look N makes N clips per scenario and look. --mode simulator also gives the generator each camera's first rendered frame, so the clip keeps the render's lighting (its look is then always Rendered).

The CLI prices the request first and shows the clips and the most it can cost, then asks before starting: type y. --yes starts without asking (needed when nothing can prompt, such as in a script), --dry-run only prices, and --max-cost-cents sets the cap yourself (by default the quote). A clip made with your prompt is labeled custom, and no check verifies its conditions.

Run your own Sim-Auto request#

simforge factory params
simforge factory clip create --raw \
  --camera camera_front_wide_120fov --control camera_front_wide_120fov=front.mp4 \
  --camera-prompt camera_front_wide_120fov="A wet city street at dusk, seen ahead from the windscreen." \
  --frames 601 --fps 30 --seed 1234 --wait
simforge factory clip create --raw --mode simulator \
  --control camera_front_wide_120fov=front.mp4 --vision camera_front_wide_120fov=front.png \
  --camera-prompt camera_front_wide_120fov="Golden hour, long shadows."

Give each camera its control video with --control CAMERA=FILE.mp4 and its prompt with --camera-prompt CAMERA=TEXT; the cameras are generated in the order you name them (--camera, else the order of --control). In simulator mode give each camera its first frame with --vision CAMERA=FILE.png (or .jpg). --prompt is the request's overall prompt. Without --frames and --aspect the request names neither, exactly as a local call: the server makes its default 201 frames, at the aspect bucket closest to your first control video.

The CLI uploads each file and checks it against what the GPU worker accepts (format, size, frame rate and frame count) before anything starts; a file that would be refused stops the request, with the reason, and nothing is charged. It then shows the price and asks:

1 raw generation: $1.00 (charged per generation, whatever it shows; no check).
Start it? Type y and press Enter to start; anything else cancels.

A raw generation costs $1.00, charged when it delivers, whatever the video shows: there is no check and no free re-seed. One that never delivers (no GPU came up, the worker failed) is free. simforge factory params shows the price this deployment charges.

Every generator option#

simforge factory params lists every option the generator has, what a scenario clip and a raw generation may set, what is fixed and why, and whether this deployment offers it. The raw generation sends the Sim-Auto option itself (the first column); a scenario clip request (POST /api/v1/factory/runs) names it by its scenario clip field. A fixed option may be given with its fixed value only, so a request you used locally carries over unchanged; any other value is refused with the reason below.

Follow, retry, cancel#

simforge factory clip list
simforge factory clip get RUN_ID
simforge factory clip wait RUN_ID
simforge factory clip retry RUN_ID --flagged
simforge factory clip cancel RUN_ID

clip get shows each clip's status, verdict, whether its prompt came from a look or from you, what it is waiting for and its cost. clip wait follows a run until every clip is done, printing what the clips wait for whenever it changes ("Waiting for a GPU, 3 clips ahead"); it exits 0 when every clip is done and non-zero when any failed. clip retry makes a run's failed and cancelled clips again (or --failed, --cancelled, --flagged for new seeds), or a failed raw generation, after the same quote and confirmation as clip create. Clips already accepted stay charged when you cancel.

Many clips from one file#

simforge factory batch clips.yaml --dry-run
simforge factory batch clips.yaml --yes --wait

- scenario: SCENARIO_ID
  look: fog-dawn
  seed: 7
- scenario: SCENARIO_ID
  prompt: Dense fog at dawn, low sun behind the haze.
  cameras: [camera_front_wide_120fov]
  mode: positions
- raw: true
  fromRequest: run_0123/request.json

Each row is its own run: a scenario row takes scenario, look or prompt, and optionally seed, cameraPrompts, cameras and mode; a raw row takes fromRequest (a request.json, below) or cameras, controls, visions, cameraPrompts, prompt, frames, fps, aspect, seed and mode. Paths are relative to the file. Every row is priced first and the total is confirmed once. Each row is submitted under a key made from what it asks, so running the same file again submits nothing new: the CLI prints the runs it already made. JSON works too (a list of the same rows).

Download#

simforge factory clip download RUN_ID --out ./clips
simforge factory clip download RUN_ID --out ./clips --links-only

The CLI writes ./clips/RUN_ID/manifest.json, every file with its signed link, sha256 and size, then fetches each file to its place and checks its sha256; running it again fetches only what is missing. --links-only writes just the manifest, for your own downloader. Nothing is zipped. A scenario clip's files are in SCENARIO_ID/generated/LOOK-SEED/: every file its manifest.json lists, among them video/CAMERA.mp4, layout/CAMERA.mp4, inputs/CAMERA.mp4 and inputs/CAMERA.png (what the generator was given), labels/boxes_2d.parquet, labels/actors_3d.parquet, labels/ego.parquet and quality.json. A raw generation's are request.json, response.mp4 (the server's own output), video/CAMERA.mp4, timing.json and inputs/ (the files it was sent). Every output is a standard format: MP4 video, JSON, and Parquet labels.

Your request, here or there#

A raw generation is exactly a local Sim-Auto call, in both directions.

From the cloud to your own server. A delivered request.json holds the request as it was sent: the form fields under request.fields, and the files under request.files in order, each read from inputs/ beside it. Start Sim-Auto with the model card's server:

vllm serve nvidia/Cosmos3-Nano-Sim-Auto --omni --model-class-name Cosmos3NanoSimAutoPipeline \
  --deploy-config <vllm-omni>/vllm_omni/deploy/cosmos3_nano_sim_auto.yaml --port 8000

and send it the delivered request unchanged, from the folder clip download wrote:

cd ./clips/RUN_ID
eval "curl -sS --fail -o local.mp4 http://127.0.0.1:8000/v1/videos/sync $(jq -r '
  [ (.request.fields | to_entries[] | "--form-string", "\(.key)=\(.value | if type == "string" then . else tojson end)"),
    (.request.files[] | "-F", "input_references=@\(.path);type=\(.mediaType)") ]
  | map(@sh) | join(" ")' request.json)"

Each field becomes a --form-string part (taken literally) and each file an input_references part, in the request's order.

From your server to the cloud. A request you used locally runs on SimForge's GPU as it is:

simforge factory clip create --raw --from-request request.json

It reads a delivered request.json, or your own {"endpoint": "/v1/videos/sync", "fields": {...}, "files": [{"filename": "front.mp4", "path": "front.mp4"}]} with fields exactly as you send them to the server (values as text, extra_params as its JSON text or the object) and files in input_references order, each path relative to the request.json's folder. The CLI uploads the files in order, checks them, and sends fields byte for byte.

Over the API#

Every command is an /api/v1 call (the command reference lists every flag): GET /api/v1/factory/params and /api/v1/factory/looks; POST /api/v1/factory/estimate, then POST /api/v1/factory/runs with maxCostCents and an Idempotency-Key; for a raw generation, POST /api/v1/uploads for each file, POST /api/v1/factory/inputs/check, then POST /api/v1/factory/generations with the request's fields, its files and maxCostCents. Retrying a create with the same Idempotency-Key returns the run it already made.