// Sensors

Example rigs

Three complete sensor rigs, each rendered end to end on a public map - a front camera, a seven-camera surround rig, and a camera with lidar and radar.

View as Markdown

Each rig below passes simforge rig validate and was rendered with the runtime on easterbrook-discovery-school, exactly as shown, on an RTX 5080. Copy one into a file, change what you need, and check it with simforge rig show. All three use the vehicle frame of a car: x forward, y up, z left, in metres from the centre of the car's box at ground level.

Front camera#

One forward camera on the windscreen, 1280x720 at 20 fps with a 90 degree field of view, tilted 3 degrees down (pitchRad -0.0524).

json

{
  "schema": "simforge.render-rig/v1",
  "sources": [
    {
      "actorId": "ego", "sensorId": "cam-front", "sensorLabel": "Front camera", "outputName": "cam-front", "modality": "rgb",
      "transform": {"position": {"x": 1.6, "y": 1.45, "z": 0}, "rotation": {"yawRad": 0, "pitchRad": -0.0524, "rollRad": 0}},
      "attributes": {"width": 1280, "height": 720, "fps": 20, "horizontalFovDeg": 90, "nearM": 0.05, "farM": 1000}
    }
  ]
}

simforge render s.json --mode 3d --rig front-camera.rig.json --out out-front

Every camera writes four passes per frame unless you narrow them with --passes: the image, metric depth, semantic classes and instance ids (here colourised for display; the files hold the raw values, see sensor outputs).

Front camera at 5 s: the RGB image, depth, semantic classes and instance ids

Seven-camera surround rig#

A layout like the camera arrays of common driving datasets: a wide and a telephoto camera looking forward, two front-corner cameras at 60 degrees, two rear-corner cameras at 120 degrees, and one camera looking back. Together they cover the whole horizon; simforge rig show reports "coveredDeg": 360.0 and no gaps.

json

{
  "schema": "simforge.render-rig/v1",
  "sources": [
    {
      "actorId": "ego", "sensorId": "cam-front", "sensorLabel": "Front wide", "outputName": "cam-front", "modality": "rgb",
      "transform": {"position": {"x": 1.7, "y": 1.5, "z": 0}, "rotation": {"yawRad": 0, "pitchRad": 0, "rollRad": 0}},
      "attributes": {"width": 1280, "height": 720, "fps": 20, "horizontalFovDeg": 120, "nearM": 0.05, "farM": 1000}
    },
    {
      "actorId": "ego", "sensorId": "cam-front-tele", "sensorLabel": "Front telephoto", "outputName": "cam-front-tele", "modality": "rgb",
      "transform": {"position": {"x": 1.72, "y": 1.52, "z": 0}, "rotation": {"yawRad": 0, "pitchRad": 0, "rollRad": 0}},
      "attributes": {"width": 1280, "height": 720, "fps": 20, "horizontalFovDeg": 30, "nearM": 0.05, "farM": 1000}
    },
    {
      "actorId": "ego", "sensorId": "cam-front-left", "sensorLabel": "Front left", "outputName": "cam-front-left", "modality": "rgb",
      "transform": {"position": {"x": 1.5, "y": 1.46, "z": 0.45}, "rotation": {"yawRad": 1.0472, "pitchRad": 0, "rollRad": 0}},
      "attributes": {"width": 1280, "height": 720, "fps": 20, "horizontalFovDeg": 120, "nearM": 0.05, "farM": 1000}
    },
    {
      "actorId": "ego", "sensorId": "cam-front-right", "sensorLabel": "Front right", "outputName": "cam-front-right", "modality": "rgb",
      "transform": {"position": {"x": 1.5, "y": 1.46, "z": -0.45}, "rotation": {"yawRad": -1.0472, "pitchRad": 0, "rollRad": 0}},
      "attributes": {"width": 1280, "height": 720, "fps": 20, "horizontalFovDeg": 120, "nearM": 0.05, "farM": 1000}
    },
    {
      "actorId": "ego", "sensorId": "cam-rear-left", "sensorLabel": "Rear left", "outputName": "cam-rear-left", "modality": "rgb",
      "transform": {"position": {"x": -0.6, "y": 1.44, "z": 0.45}, "rotation": {"yawRad": 2.0944, "pitchRad": 0, "rollRad": 0}},
      "attributes": {"width": 1280, "height": 720, "fps": 20, "horizontalFovDeg": 90, "nearM": 0.05, "farM": 1000}
    },
    {
      "actorId": "ego", "sensorId": "cam-rear-right", "sensorLabel": "Rear right", "outputName": "cam-rear-right", "modality": "rgb",
      "transform": {"position": {"x": -0.6, "y": 1.44, "z": -0.45}, "rotation": {"yawRad": -2.0944, "pitchRad": 0, "rollRad": 0}},
      "attributes": {"width": 1280, "height": 720, "fps": 20, "horizontalFovDeg": 90, "nearM": 0.05, "farM": 1000}
    },
    {
      "actorId": "ego", "sensorId": "cam-rear", "sensorLabel": "Rear", "outputName": "cam-rear", "modality": "rgb",
      "transform": {"position": {"x": -2.2, "y": 1.2, "z": 0}, "rotation": {"yawRad": 3.14159, "pitchRad": 0, "rollRad": 0}},
      "attributes": {"width": 1280, "height": 720, "fps": 20, "horizontalFovDeg": 90, "nearM": 0.05, "farM": 1000}
    }
  ]
}

Camera	Faces	yawRad	Field of view
`cam-front`	forward	0	120 degrees
`cam-front-tele`	forward	0	30 degrees
`cam-front-left`	front left	1.0472 (60 degrees)	120 degrees
`cam-front-right`	front right	-1.0472 (-60 degrees)	120 degrees
`cam-rear-left`	rear left	2.0944 (120 degrees)	90 degrees
`cam-rear-right`	rear right	-2.0944 (-120 degrees)	90 degrees
`cam-rear`	back	3.14159 (180 degrees)	90 degrees

The rear camera's yaw is 3.14159, inside the allowed range of -pi to pi; 3.1416 is just outside it and fails validation.

simforge render s.json --mode 3d --rig surround-7cam.rig.json --preset training --out out-surround

The seven cameras at 5 s, training preset: front left, front, front right, rear left (top row); front telephoto, rear right, rear (bottom row)

Camera, lidar and radar#

The front camera, a 64-channel roof lidar and a forward radar 0.2 m ahead of the front of the box. The lidar spins at 10 Hz with a symmetric vertical field of +15 to -15 degrees and 1.2 million points per second, so each scan has up to 64 x 1875 points. The radar casts 1500 rays per second over a 90 x 10 degree field. A rig with a radar needs video: its fps (20 here) is the radar's sample rate.

json

{
  "schema": "simforge.render-rig/v1",
  "sources": [
    {
      "actorId": "ego", "sensorId": "cam-front", "sensorLabel": "Front camera", "outputName": "cam-front", "modality": "rgb",
      "transform": {"position": {"x": 1.6, "y": 1.45, "z": 0}, "rotation": {"yawRad": 0, "pitchRad": -0.0524, "rollRad": 0}},
      "attributes": {"width": 1280, "height": 720, "fps": 20, "horizontalFovDeg": 90, "nearM": 0.05, "farM": 1000}
    },
    {
      "actorId": "ego", "sensorId": "lidar-roof", "sensorLabel": "Roof lidar", "outputName": "lidar-roof", "modality": "lidar",
      "transform": {"position": {"x": 0, "y": 1.9, "z": 0}},
      "attributes": {"channels": 64, "rangeM": 120, "pointsPerSecond": 1200000, "rotationFrequencyHz": 10, "upperFovDeg": 15, "lowerFovDeg": -15, "horizontalFovDeg": 360}
    },
    {
      "actorId": "ego", "sensorId": "radar-front", "sensorLabel": "Front radar", "outputName": "radar-front", "modality": "radar",
      "transform": {"position": {"x": 2.6, "y": 0.5, "z": 0}},
      "attributes": {"horizontalFovDeg": 90, "verticalFovDeg": 10, "rangeM": 150, "pointsPerSecond": 1500}
    }
  ],
  "video": {"width": 1280, "height": 720, "fps": 20, "container": "mp4", "codec": "h264", "quality": "standard"}
}

simforge render s.json --mode 3d --rig camera-lidar-radar.rig.json --out out-lidar

The render writes the camera's passes, one .ply point cloud for the lidar and one .csv of detections for the radar at every frame, and a review video for each sensor.

The camera, and the lidar and radar review videos at 5 s

Mount a radar outside the vehicle's body. A radar inside it, for example at the box's front edge where the body's nose is, sees only the body: every detection is a few centimetres away.

The scenario used#

simforge maps pull easterbrook-discovery-school --accept-license
simforge scenario new --map easterbrook-discovery-school --clip-seconds 10 --out s.json
simforge scenario apply s.json add-actor --id ego --catalog-id vehicle.sedan --lane-rsl 10:0:-1 --s 6 \
  -- set-ego --role-id ego -- add-actor --id lead --catalog-id vehicle.sedan --lane-rsl 10:0:-1 --s 30 -- set-clock --minutes 780

A sedan ego following a second sedan down road 10 at about 13 m/s, at 13:00, for 10 seconds: 200 frames at 20 fps.