Preview and render

One renderer, the Helios Engine, in three modes:

Pick a sensor rig#

Schematic and 3D renders look through a rig: the cameras on the vehicle. Without --rig they use the basic-dash-camera rig on the ego, a front camera at the preset's capture (1280x720 at 24 fps for the default showcase); write your own to choose others.

simforge rig init --preset basic-dash-camera --out rig.json
simforge rig show rig.json

rig init writes the cameras at the --profile capture: showcase (1280x720 at 24 fps) unless you pass --profile training (512x384 at 20 fps). The basic dash camera's camera is dashcam-front. Other presets include alpamayo-2cam, alpamayo-6cam, tesla-hw3 and waymo-5th-gen; simforge rig init --help lists them all, and the vehicle classes.

Preview a frame#

simforge preview frame s.json --mode 2d --at 5 --out plan.png
simforge preview frame s.json --mode 3d --rig rig.json --camera dashcam-front --at 5 --out frame.png
simforge preview clip s.json --mode 3d --rig rig.json --camera dashcam-front --at 5 --seconds 3 --out clip.mp4

Render#

simforge render s.json --mode 2d --out out-2d
simforge render s.json --mode schematic --rig rig.json --out out-schematic
simforge render s.json --mode 3d --rig rig.json --out out-3d

--out is a folder that must not exist or be empty. 3D renders take --preset showcase or --preset training (the dataset look at a fixed cost per frame); without --preset, a 3D render uses showcase, locally and with --cloud. They also take --passes to choose among rgb, id, depth and semantic.

GPU memory and texture tiers#

How much GPU memory a 3D render needs depends on the map and the texture tier much more than on the cameras. The CLI picks the tier from your GPU: full textures (uastc-full) on 16 GiB and more, 512 px textures (bc7-512) below that, and says so in the result (localDefaults). --textures uastc-full or --textures bc7-512 overrides it. On an Apple silicon Mac the GPU memory is three quarters of the Mac's memory: a 24 GB Mac gets full textures, a 16 GB Mac 512 px textures.

Peak GPU memory of one 3D render (showcase preset, one 1280x720 camera unless stated):

The training preset needs about the same; a schematic render needs 0.2 GiB. Measured on an RTX 5080 (driver 595.84). The machine-readable table is sdk/contracts/runtime-support.v1.json in the toolkit.

Output format#

A 3D render writes:

Ticks count from the clip start at the rig's frame rate. Lidar sources write <sensor>/<tick:08>.ply and radar sources .csv. A 2D render writes render.json and video/topdown.mp4. Traces are identical on every machine; pixels are identical run to run on the same GPU model and driver, and differ slightly between GPUs.

Without ffmpeg, a 2D render that requests video preserves its images under frames/topdown/<index:08>.png and reports that directory in frame.directory; it warns that no video was encoded. --video off remains a render-and-measure operation without retained frames.

Before you render#

Check the GPU with simforge doctor and the runtime's render modes with simforge runtime status. A render that cannot run on this machine fails with a named error; it never falls back to another device or to the cloud. A scene that does not fit in GPU memory fails with native_gpu_out_of_memory: use --textures bc7-512, fewer or smaller cameras, or --cloud. A 3D render whose GPU memory cannot be measured warns gpu_memory_unmeasured; --vram-budget sets the budget explicitly.

Render in the cloud#

The same command runs on SimForge's machines with --cloud and a cost cap in cents:

simforge render s.json --mode 3d --rig rig.json --out out-3d --cloud --max-cost 200 --wait --download

Cloud downloads include every captured sensor file named by results.json, the videos and output metadata. The local renderer's diagnostic job/ directory is not exported in full; its authoritative job/timeline.json is included.

See Run in the cloud for pricing, caps and following a run.