// Reference

Model host contract

The versioned wire a SimForge closed-loop evaluation speaks to a driving model served over HTTP (simforge.model-host/v1), and a conformance command that checks your own server against it in one step.

View as Markdown

A closed-loop evaluation renders cameras live and asks a driving model, over HTTP, what to do next. The model sits behind four calls: GET /healthz, POST /v1/sessions, POST /v1/sessions/{id}/invoke and DELETE /v1/sessions/{id}. This page is that contract, version simforge.model-host/v1. The same wire serves SimForge's own hosted models, so a server that passes the check below speaks what they speak.

Serving your own model on SimForge is not available yet. This is the first step: prove your server speaks the contract, on your own machine, with no account and no cost.

Check your server in one command#

pip install simforge-gym
export SIMFORGE_MODEL_HOST_TOKEN=...        # any value if your server ignores the bearer
python -m simforge_gym.closed_loop.conformance --url http://127.0.0.1:8765

The command acts as a closed-loop episode against your server, with the client and the readers of your answers that the episode itself uses. It reads /healthz, opens a session, sends decisions with the frame window, ego history and route your declaration asks for, times every answer, probes how you refuse a malformed decision, and closes the session. The frames are synthetic gray JPEGs and the ego drives straight at 5 m/s, so it needs no scenario, rig or account.

text

PASS health.reachable: GET /healthz answered with a JSON object
FAIL health.observation: capabilities.observation.frameHistoryOffsetsS must end at 0, not -0.1: the newest frame is the decision instant
SKIP session.open: the declaration failed; the episode would refuse the run before opening a session
FAIL: simforge.model-host/v1, 1 failed, 0 warnings, 3 passed

Each check ends pass, warn, fail or skipped. A fail is something a SimForge episode would refuse, or one of the v1 rules listed under Conformance requirements and loop tolerances. A warn is something the loop tolerates today and a hosted run should not depend on (a missing reasoning, an unknown field accepted). skipped means an earlier failure made it moot. The exit status is 0 without a failure, 1 with one (also on a warning under --strict) and 2 when the command was misused.

Flag	Meaning
`--url URL`	Your server's base URL, `http` or `https`.
`--token-env NAME`, `--token-file FILE`	Where the bearer token comes from (default `SIMFORGE_MODEL_HOST_TOKEN`). It is never printed and never a command-line argument.
`--deadline-ms N`	The longest an invoke may take. Default 600000, the limit the closed loop itself applies to one request.
`--decisions N`	Decisions to send (default 3).
`--replan-every N`	Steps of 0.1 s between decisions (default 5, the episode default).
`--camera-ids 0,1,2`	Camera indices to open the session with, when your declaration does not fix them.
`--frame-size WxH`	Size of the synthetic frames (default 1920x1080, the native rig).
`--rig FILE`	A fitted rig. Checked equal to `capabilities.rig.definition` when you declare one.
`--healthz-file FILE`	Check only a saved `/healthz` document; no server needed.
`--json`, `--strict`	One `simforge.model-host-conformance/v1` document on stdout; fail on warnings.

python -m simforge_gym.closed_loop.reference_host --port 8765 starts the smallest server that passes: a fake model that answers every decision with the same straight-ahead trajectory. Start from it when you wire your own model in.

Versioning and the schema#

The contract is simforge.model-host/v1. A change that makes a valid v1 server invalid is a new version; new optional fields are additions to v1. The messages are published as JSON Schema (draft 2020-12), model-host.v1.schema.json, packaged with simforge-gym beside the conformance command. It has one definition per message: Health, OpenSessionRequest, OpenSessionResponse, InvokeRequest, InvokeResponse and Refusal. Camera JPEGs are bytes on the MessagePack wire; the schema describes them as base64 strings only to stay JSON.

Transport#

Rule	Detail
Authentication	Every request carries `Authorization: Bearer <token>`. A host verifies it and answers a wrong or missing one with HTTP 401.
Encodings	`/healthz`, session creation and the refusals are JSON. The invoke request is MessagePack (`application/msgpack`). The invoke answer must be `application/msgpack`.
One host	A run contacts only the host it was configured with. A failure is never retried and never answered by another model.
Unknown fields	A host refuses unknown fields in a session or invoke request, so the names on this page are exact.

`GET /healthz`: the declaration#

The loop sends the model exactly what the host declares, renders with the settings it declares and adopts its plans after the delays it declares. There is no guessing: v1 requires the declaration, and hosted evaluations require capabilities.observation before rendering. Local simforge-gym may use a registered family's observation contract and record observation_contract_undeclared; an undeclared unknown family is refused with host_observation_undeclared.

Field	Required	Meaning
`ok`	yes	`true`.
`family`	yes	Nonempty name of the model family.
`camera_profile`	yes	The rig preset SimForge renders for the model: `alpamayo-4cam` (cameras 0, 1, 2, 6), `alpamayo-6cam` (0, 1, 2, 3, 5, 6) or `qwen-drive-3cam` (0, 1, 2).
`horizon_s`	no	Seconds a plan covers (default 6.4). Positive and finite.
`samples.session`	no	Trajectories returned per decision (default 1). The loop drives the first.
`samples.openLoop`	no	Native samples of open-loop evaluation, the `numTrajSamples` of the model registry (wire parameter `params.num_traj_samples`).
`capabilities.observation`	yes	What the loop feeds the model; see below.
`capabilities.cameras.required`	no	Camera indices; must equal the profile's.
`capabilities.rig.definition`	no	The calibrated rig the model was trained on. When present, the rendered rig must equal it or the run is refused (`model_rig_mismatch`).

capabilities.observation:

Field	Default	Rule
`frameHistoryOffsetsS`	none, required	Nonempty list of seconds before the decision, oldest first: each 0 or less, on the 0.1 s grid, strictly increasing, last exactly 0. `[-0.3, -0.2, -0.1, 0]` is four frames per camera at 10 Hz; `[-1.5, -1.0, -0.5, 0]` four frames over 1.5 s. Every camera sends the same instants.
`frameEncoding`	`jpeg`	Only `jpeg`.
`requiresRoute`	`false`	`true` adds `route_xyz` to every decision.
`egoHistoryS`	`1.5`	Seconds of ego history, a multiple of 0.1. The history holds `egoHistoryS / 0.1 + 1` poses (16 by default).
`planAdoptDelayS`	`{first: 0, later: 0}`	Both keys, seconds on the 0.1 s grid. `first` must be 0: the first plan steers at once. `later` must be shorter than the replanning interval (0.5 s by default).
`renderOverrides`	`{}`	Only `{"lens.distortion": 0}`: render without lens distortion.

`POST /v1/sessions`#

Request (JSON): {"camera_ids": [0, 1, 2, 6], "seed": 0}, the cameras of the declared profile and an integer seed. A host that does not serve those cameras refuses with camera_set_invalid.

Answer (JSON): {"ok": true, "sessionId": "...", "model": {...}, "gpu": {...}}. model carries four nonempty strings, family, revision, checkpointDigest and quant: a run can pin them (model_revision_mismatch otherwise) and records them as the model's identity. gpu is any nonempty object naming the accelerator; SimForge treats it as opaque provenance. framesPerCamera and frameHistoryOffsetsS are optional and, when present, must agree with /healthz.

`POST /v1/sessions/{id}/invoke`: one decision#

The episode asks for a decision every 0.5 s of simulated time, once enough real history exists: the declared frame history and ego history are never padded.

Request (MessagePack):

Field	Meaning
`cameras`	`[{camera_id, jpeg, t_s}]`: JPEG `bytes`. Only frames the host has not been sent yet, oldest first. The host keeps each camera's window of the declared offsets. The first decision sends the whole window; each later one sends the frames new since.
`ego_history_xyz`, `ego_history_rot`, `ego_history_t_s`	The last poses in the ego frame at `t_s` (x forward, y left, z up, origin at the footprint centre): `[x, y, z]` points, 3x3 rotations and times relative to `t_s`, 0.1 s apart, ending at 0 with the identity pose.
`t_s`	The decision instant, equal to the newest frame's `t_s`. Increases between decisions.
`seed`, `params`	The session seed and the run's parameters (`{}` by default).
`route_xyz`	With `requiresRoute`: at least 2 `[x, y, z]` points of the remaining route in the ego frame at `t_s`.

Answer (MessagePack, application/msgpack):

Field	Required	Meaning
`ok`	yes	`true`.
`trajectories`	yes	One or more plans, each a list of at least 2 finite `[x, y]` or `[x, y, z]` waypoints in the ego frame at `t_s`, `dt_s` apart, the first `dt_s` after `t_s`. Every plan covers the same number of waypoints.
`dt_s`	no	Must be `0.1`; assumed when absent.
`reasoning`	no	A list whose first entry is the model's text; without it the run records `reasoning_missing`.
`timings`, `tokens`, `rng_provenance`	no	Kept in the run's record.

A plan shorter than horizon_s is allowed (the check warns): the vehicle follows the plan it is given.

Timing#

Deadline. A request that has not been answered within the deadline fails the run with model_host_unreachable. The closed loop's limit is 600 s per request; set --deadline-ms to the budget you want to hold yourself to.
Simulated time stands still while the model thinks. An episode runs in offline simulated time: the wall time of an invoke costs the run's duration, not the vehicle's progress. The only simulated delay is the one you declare in planAdoptDelayS: a plan decided at t steers from t + later, and the first plan from t.
Order. One decision at a time per session. t_s increases; a host may refuse a decision that does not (input_error).

Errors#

A refusal is {"ok": false, "error": {"code": "...", "message": "..."}}, answered with HTTP 200 on a session call or with a 4xx status; error.fields may name the offending request fields. A host answers its own faults with a 5xx and the same body. The loop never retries.

What happened	Run fails with
The host cannot be reached, closes the connection or misses the deadline	`model_host_unreachable`
HTTP 401 or 403	`model_host_unauthorized`
An undecodable body, or a body that is not an object	`model_host_protocol_error`
`ok: false` or another HTTP error	the host's `error.code` (or `model_host_error`)
An invalid declaration	`observation_contract_invalid`
A declared rig that differs from the rendered one	`model_rig_mismatch`
No trajectories, fewer than 2 waypoints, non-finite values or a `dt_s` other than 0.1	`model_output_invalid`

Use stable, specific codes for your own refusals, such as missing_fields, input_error, camera_set_invalid, frame_window_incomplete and session_not_found: they appear in your run's failure reason.

Conformance requirements and loop tolerances#

The conformance check reports each violation below as fail. Some requirements are stricter than the local gym or the loop's decoding tolerance; hosted evaluations already require an observation declaration.

v1 requires capabilities.observation and camera_profile. Hosted evaluations require the observation declaration; only local simforge-gym may use a registered family's observation contract with a warning. An undeclared unknown family is refused.
The invoke answer is application/msgpack. The loop also decodes a JSON body.
trajectories holds equally long plans, and as many as samples.session when declared. The loop reads only the first plan.
The opened session's model has nonempty family, revision, checkpointDigest and quant. The loop requires the object and compares only the keys a run pins.

capabilities.cameras.required disagreeing with camera_profile is a warn: the loop never reads it.

The checks#

Check	What it verifies
`health.reachable`, `health.auth`	`/healthz` answers with the bearer; it is refused without one (warning if not).
`health.shape`, `health.observation`, `health.horizon`, `health.cameras`, `health.samples`, `health.rig`	The declaration, field by field, with the offending path and value in the message.
`session.open`, `session.unknown_fields`	A session opens with the declared cameras and identifies the model; an unknown field is refused.
`invoke.accepted`, `invoke.content_type`	Valid decisions are answered `ok`, as `application/msgpack`.
`invoke.trajectories`, `invoke.cadence`, `invoke.horizon`, `invoke.samples`, `invoke.reasoning`	The shape of the answer, element by element.
`invoke.deadline`	Every answer arrives within `--deadline-ms`.
`invoke.refusal`, `invoke.unknown_fields`	A decision without an ego history is refused with a code, never a 5xx or an `ok` answer; an unknown field is refused.
`session.close`	`DELETE /v1/sessions/{id}` succeeds.