# Sensor outputs

> What a render writes for each camera, lidar and radar - the folder layout, file formats, timestamps, the camera and vehicle poses - and how to load it in Python.

Source: https://simforge.ai/docs/sensor-outputs

A local 3D render writes everything into its `--out` folder, which must not exist yet or be empty. This is the layout for the [camera, lidar and radar example](/docs/sensor-examples#camera-lidar-and-radar):

```text
out-lidar/
  render.json              the render's summary: rig, preset, render settings, timings, videos, warnings
  results.json             every captured file: sensor, tick, pass, bytes, sha256, and each camera frame's projection and time
  observed-frames.jsonl    one line per frame: every actor's position, rotation and visibility as rendered
  legend.json              instance id -> the static map mesh it is
  parity.json              how closely the rendered poses match the simulated trace
  render.log
  cam-front/00000000.rgb.png          one file per frame and pass
  cam-front/00000000.depth.f32.bin
  cam-front/00000000.semantic.png
  cam-front/00000000.id.png
  lidar-roof/00000000.ply             one point cloud per frame
  radar-front/00000000.csv            one detection list per frame
  video/cam-front.mp4                 one video per sensor (lidar and radar as review videos)
  job/render-job.json                 the exact inputs, including every camera's pose at every frame
```

Folder and file names come from each source's `outputName`. Ticks are zero-padded to eight digits.

A lidar or radar review video shows the sensor's right on the right of the picture. Review videos rendered before 13 October 2026 are mirrored left to right; the point clouds and radar files of those renders are correct.

## Camera files

| File | Format |
|---|---|
| `<tick>.rgb.png` | The image, RGBA8 PNG, sRGB |
| `<tick>.depth.f32.bin` | Little-endian float32, width x height, row-major, reverse-Z: distance along the camera axis in metres = `nearM / value`; 0 means nothing was hit (sky) |
| `<tick>.semantic.png` | The class in the blue channel of an RGBA8 PNG, CARLA CityScapes ids: 0 unlabeled, 1 building, 2 fence, 4 pedestrian, 5 pole, 7 road, 9 vegetation, 10 vehicle, 12 traffic sign, 18 traffic light |
| `<tick>.id.png` | The instance id, RGBA8 PNG: id = R + 256 x G + 65536 x B, 0 = sky or background. `legend.json` names the static map meshes; ids missing from it are actors |

`--passes rgb,depth` (any of `rgb`, `id`, `depth`, `semantic`) writes only the passes you need. A 1280x720 frame takes about 1 MB as RGB and 3.7 MB as depth, so 7 cameras x 200 frames with all four passes come to 6.3 GB.

The RGB image carries the preset's camera look: exposure, tone mapping, bloom, a vignette and a slight lens distortion (0.008). Depth, id and semantic are drawn without them, so their pixels line up with an undistorted pinhole. For pixel-exact geometry in the RGB image too, turn the distortion off: `--set lens.distortion=0`. `render.json` records every setting in `renderConfig`.

## Lidar files

`<tick>.ply` is an ASCII PLY point cloud with one row per point:

```text
ply
format ascii 1.0
element vertex 91524
property float x
property float y
property float z
property float intensity
property uint instance_id
end_header
```

`x`, `y` and `z` are metres in the lidar's own frame: x forward along the sensor's line of sight, y up, and **z to the right** (unlike the rig's vehicle frame, whose z points left). `intensity` is a reflectance proxy in 0 to 1, and `instance_id` uses the same ids as the cameras' id pass.

The renderer writes one full scan at every rendered frame, the camera rate: the example's 10 Hz lidar gives 200 point clouds in 10 seconds at 20 fps, not 100. Every point of a scan is taken at the frame's instant (no motion during the sweep), and `rotationFrequencyHz` sets how dense a scan is: each ring has pointsPerSecond / (channels x rotationFrequencyHz) samples, kept between 64 and 4096. Beams that hit nothing within `rangeM` return no point, so the count varies from frame to frame.

## Radar files

`<tick>.csv` has one detection per row:

| Column | Unit | Meaning |
|---|---|---|
| `depth_m` | metres | Range along the beam |
| `azimuth_rad` | radians | Beam direction; 0 is the radar's forward axis, positive to the right |
| `altitude_rad` | radians | Beam elevation; positive up |
| `velocity_mps` | m/s | Radial velocity relative to the radar; negative when the target closes in. Static objects ahead of a vehicle driving at 13 m/s read about -13 |

## Time and synchronization

Every sensor is captured at the same instants. Frame `n` is at time clipStart + n / fps, in seconds from the scenario's start: `results.json` gives each camera frame's `sampleTime.seconds`, and `observed-frames.jsonl` each frame's `time`. The rig's `clip` sets clipStart (0 by default). A frame is an instantaneous snapshot: rolling shutter, exposure time and sensor clock drift are not modelled (`results.json` says so per frame under `camera.sampling`).

## Poses and calibration

There is no separate calibration file: the rig you rendered with is the calibration.

- **Intrinsics.** A pinhole camera has fx = fy = (width / 2) / tan(horizontalFovDeg / 2) and its principal point at the image centre. `results.json` gives each frame's `projection`: `kind`, `fovYDeg`, `nearM`, `farM` and the depth convention.
- **Extrinsics.** Each source's `transform` in the rig: its position and rotation on the vehicle.
- **The vehicle in the world.** `observed-frames.jsonl` gives every actor's pose at every frame: `position` `[x, y, z]` in metres and `rotation` as a quaternion `[x, y, z, w]`, in the map's scene frame (x east, y up, z south).
- **Each camera in the world.** `job/render-job.json` lists every camera at every frame (`rig.cameraSchedule`): its mount as the renderer applied it (`attach`), and its position (`eye`) and a point 50 m along its line of sight (`target`) in the scene frame.

## Load it in Python

The toolkit has no reader for these files; NumPy and Pillow are enough:

```python
import json
from pathlib import Path

import numpy as np
from PIL import Image

out = Path("out-lidar")
results = json.loads((out / "results.json").read_text())

# Every captured file: its sensor, tick, pass and, for cameras, the projection and sample time.
for artifact in results["artifacts"][:3]:
    print(artifact["sensorId"], artifact["tick"], artifact["pass"], artifact["path"])

# One camera frame: the image, instance ids, semantic classes and metric depth.
cam, tick = "cam-front", 40
rgb = np.asarray(Image.open(out / cam / f"{tick:08d}.rgb.png"))[..., :3]
idx = np.asarray(Image.open(out / cam / f"{tick:08d}.id.png")).astype(np.uint32)
instance = idx[..., 0] + 256 * idx[..., 1] + 65536 * idx[..., 2]
semantic = np.asarray(Image.open(out / cam / f"{tick:08d}.semantic.png"))[..., 2]
height, width = rgb.shape[:2]
raw = np.fromfile(out / cam / f"{tick:08d}.depth.f32.bin", dtype="<f4").reshape(height, width)
near_m = 0.05  # the camera's nearM
depth_m = np.where(raw > 0, near_m / np.maximum(raw, 1e-12), np.inf)  # inf: nothing hit

# Pinhole intrinsics from the rig: horizontalFovDeg 90 at 1280x720.
fx = fy = (width / 2) / np.tan(np.radians(90) / 2)
cx, cy = width / 2, height / 2

# One lidar scan: x forward, y up, z right of the sensor, metres.
def read_ply(path):
    lines = path.read_text().splitlines()
    start = lines.index("end_header") + 1
    return np.loadtxt(lines[start:], ndmin=2)  # columns: x y z intensity instance_id

points = read_ply(out / "lidar-roof" / "00000004.ply")

# One radar frame: range, azimuth, elevation, radial velocity.
radar = np.genfromtxt(out / "radar-front" / "00000040.csv", delimiter=",", names=True)
```

## In the cloud

A cloud 3D render (`--cloud`) returns each sensor's video (`video/<sensor>.mp4`, lidar and radar as review videos) with `render.json` and `results.json`. It does not return the per-frame images, depth, id or semantic passes, point clouds or radar files: render locally when you need them. `simforge preview frame --cloud` returns the RGB frame as a PNG.