Custom sensor rigs

A sensor rig is a JSON file that lists what a render captures: the cameras, lidars and radars, where each one sits on the vehicle, which way it looks and what it records. The format is simforge.render-rig/v1. A rig is not part of a scenario: you pass it to each render, so one rig works with every scenario, and one scenario can be rendered through many rigs.

Schematic and 3D renders look through a rig. Without --rig they use the built-in basic-dash-camera on the scenario's ego. 2D renders are a top-down map view and take no rig.

The quick path#

simforge rig init --preset basic-dash-camera --class car --out rig.json
simforge rig validate rig.json
simforge rig show rig.json
simforge render s.json --mode 3d --rig rig.json --out out-3d

rig init writes a built-in rig fitted to a vehicle class's default box, which is the easiest starting point to edit. rig validate checks the file exactly as a render reads it and exits 2 with the problem. rig show prints what the rig captures: each sensor's mount, the direction it faces, its field of view, and how much of the horizon the cameras cover together, with any gaps. Every field is in the rig file reference, and three complete, rendered rigs are in example rigs.

How a rig attaches to the vehicle#

Every scenario names an ego: the vehicle that carries the sensors (a car, truck, bus, van, motorcycle or drone). A render mounts every source of the rig on that ego, whatever actorId the file names, and says so with the warning render_rig_rehosted when it moves one. So actorId: "ego" in a rig works for any scenario. A scenario without an ego is refused (render_ego_missing).

Mounts are rigid: a sensor moves with the vehicle body, including its pitch and roll over bumps and on slopes, and it never sees the vehicle that carries it. The one exception is a drone camera with mountFrame: "gimbal", whose rotation follows the drone's stabilized gimbal while its position rides the body.

The vehicle frame#

Positions are metres in the vehicle's own frame. The origin is the centre of the vehicle's box at ground level.

 Top view (looking down)                 Side view (from the left)

            +X forward                              +Y up
                ^                                     ^
                |                                     |      o-> camera, looking +X
         +------+------+                        +-----+----------+
         |      |      |                        |     |          |
 +Z <----+------o      |                        |     |          |
 left    |   vehicle   |                     ---+-----o----------+---> +X forward
         |    box      |                        ground (y = 0); o = origin
         +-------------+
            o = origin (box centre, at ground level)

X forward, Y up and Z left make a left-handed frame. Keep that in mind when you bring mounts from a right-handed convention: see converting calibrations below.

Default boxes, used to fit the built-in rigs: car 4.8 x 1.9 x 1.5 m, van 5.5 x 2.0 x 2.2 m, truck 9.5 x 2.5 x 3.5 m, drone 1.0 x 1.0 x 0.45 m (length x width x height). A scenario actor with its own dimensions keeps its own box, and your rig's positions are used as written.

Rotations#

A sensor's rotation is three angles in radians, all optional and 0 by default. At zero rotation a sensor looks straight ahead along +X, level, with its image upright.

They apply in that order: yaw first, then pitch about the turned sensor, then roll about its own line of sight. There is no quaternion or matrix input. Worked examples:

A rear camera's yaw must stay inside the range: write 3.14159 or -3.14159, not 3.1416, which rig validate refuses (transform.rotation is outside yaw/roll [-pi, pi], pitch [-pi/2, pi/2]).

Converting a calibration you already have#

Many calibrations use a right-handed vehicle frame with x forward, y left and z up (FLU), or x forward, y right and z down (FRD). To move a mount into a SimForge rig:

If your origin is somewhere else, such as the rear axle, subtract that point's position in the SimForge frame first: for a rear axle 1.4 m behind the box centre, x = x_axle - 1.4. Check every converted rig with simforge rig show: its facing column names the side each sensor looks at (front, front-left, left, rear-left and so on), which catches a flipped sign at once.

Render profiles#

The --preset flag picks how a 3D render looks; the rig picks what it captures.

Without --rig, the default dash camera gets the profile's capture size and rate. rig init writes its cameras at a profile's capture (--profile, default showcase): fit a rig to the profile you render with, such as --profile training for a training render. A rig file keeps its own: each camera renders at its own width, height and fps whatever the profile. Give every camera of a rig the same fps; cameras at different rates are scheduled on the union of their frame times.

Use a rig#

Locally, with the runtime installed:

simforge rig validate rig.json
simforge preview frame s.json --mode 3d --rig rig.json --camera cam-front --at 5 --out frame.png
simforge render s.json --mode 3d --rig rig.json --out out-3d
simforge render s.json --mode 3d --rig rig.json --preset training --passes rgb,depth --out out-train

preview frame renders one camera at one moment, which is the fastest way to check a mount; --camera takes a camera's outputName. --passes chooses the camera passes (rgb, id, depth, semantic; all four by default).

In the cloud, the same file goes with the request:

simforge render s.json --mode 3d --rig rig.json --out out-3d --cloud --dry-run
simforge render s.json --mode 3d --rig rig.json --out out-3d --cloud --max-cost 100 --wait --download

A cloud render sends the rig as part of the run (the rig of a render run in the Cloud API) and renders it with the same runtime. It returns each sensor's video plus render.json and results.json, not the per-frame images, depth or point clouds: render locally when you need those. See sensor outputs.

In Studio, the Render panel's sensor step offers the built-in rigs, a single front camera and the rig this scenario last rendered with. Studio has no rig upload yet: to render your own rig file, use the CLI, locally or with --cloud.

Reuse a rig across scenarios#

A rig names no map, scenario or actor of its own, so keep one file per sensor setup and pass it to every render:

for s in scenarios/*.json; do
  simforge render "$s" --mode 3d --rig rigs/surround.rig.json --out "out/$(basename "$s" .json)"
done

Positions are absolute metres on the vehicle, so a rig written for a car sits in the same place on a truck: the cameras are not moved to the truck's roof. Write one rig per vehicle class you render, or start each from simforge rig init --preset <preset> --class <class>, which fits a built-in rig to the class's box.

What a rig cannot describe yet#

These are not supported today. A field for any of them is refused as unknown:

• Pinhole intrinsics as fx, fy, cx, cy, or a shifted principal point: a pinhole camera is horizontalFovDeg with square pixels and the principal point at the image centre.
• Brown-Conrady (OpenCV radial-tangential), Kannala-Brandt or equidistant fisheye distortion: the only lens model is the f-theta polynomial (cameraModel).
• Exposure, shutter speed, rolling shutter, motion blur per camera, noise or lens flare settings.
• IMU, GNSS, ultrasonic, thermal or event cameras. The vehicle's pose at every frame is in observed-frames.jsonl.
• Separate depth, semantic or instance camera sources: those are passes of an RGB camera (--passes); a source with modality depth, semantic or instance validates but the renderer refuses it (native_modality_unsupported).
• A lidar whose vertical field is not symmetric about its mount (for example +15 to -25 degrees), lidar noise or dropout, and radar noise or clutter.