# Model host contract

> The versioned wire a SimForge closed-loop evaluation speaks to a driving model served over HTTP (simforge.model-host/v1), and a conformance command that checks your own server against it in one step.

Source: https://simforge.ai/docs/model-host

A closed-loop evaluation renders cameras live and asks a driving model, over HTTP, what to do next. The model sits behind four calls: `GET /healthz`, `POST /v1/sessions`, `POST /v1/sessions/{id}/invoke` and `DELETE /v1/sessions/{id}`. This page is that contract, version `simforge.model-host/v1`. The same wire serves SimForge's own hosted models, so a server that passes the check below speaks what they speak.

Serving your own model on SimForge is not available yet. This is the first step: prove your server speaks the contract, on your own machine, with no account and no cost.

## Check your server in one command

```sh
pip install simforge-gym
export SIMFORGE_MODEL_HOST_TOKEN=...        # any value if your server ignores the bearer
python -m simforge_gym.closed_loop.conformance --url http://127.0.0.1:8765
```

The command acts as a closed-loop episode against your server, with the client and the readers of your answers that the episode itself uses. It reads `/healthz`, opens a session, sends decisions with the frame window, ego history and route your declaration asks for, times every answer, probes how you refuse a malformed decision, and closes the session. The frames are synthetic gray JPEGs and the ego drives straight at 5 m/s, so it needs no scenario, rig or account.

```text
PASS health.reachable: GET /healthz answered with a JSON object
FAIL health.observation: capabilities.observation.frameHistoryOffsetsS must end at 0, not -0.1: the newest frame is the decision instant
SKIP session.open: the declaration failed; the episode would refuse the run before opening a session
FAIL: simforge.model-host/v1, 1 failed, 0 warnings, 3 passed
```

Each check ends `pass`, `warn`, `fail` or `skipped`. A `fail` is something a SimForge episode would refuse, or one of the v1 rules listed under [Conformance requirements and loop tolerances](#conformance-requirements-and-loop-tolerances). A `warn` is something the loop tolerates today and a hosted run should not depend on (a missing `reasoning`, an unknown field accepted). `skipped` means an earlier failure made it moot. The exit status is 0 without a failure, 1 with one (also on a warning under `--strict`) and 2 when the command was misused.

| Flag | Meaning |
|---|---|
| `--url URL` | Your server's base URL, `http` or `https`. |
| `--token-env NAME`, `--token-file FILE` | Where the bearer token comes from (default `SIMFORGE_MODEL_HOST_TOKEN`). It is never printed and never a command-line argument. |
| `--deadline-ms N` | The longest an invoke may take. Default 600000, the limit the closed loop itself applies to one request. |
| `--decisions N` | Decisions to send (default 3). |
| `--replan-every N` | Steps of 0.1 s between decisions (default 5, the episode default). |
| `--camera-ids 0,1,2` | Camera indices to open the session with, when your declaration does not fix them. |
| `--frame-size WxH` | Size of the synthetic frames (default 1920x1080, the native rig). |
| `--rig FILE` | A fitted rig. Checked equal to `capabilities.rig.definition` when you declare one. |
| `--healthz-file FILE` | Check only a saved `/healthz` document; no server needed. |
| `--json`, `--strict` | One `simforge.model-host-conformance/v1` document on stdout; fail on warnings. |

`python -m simforge_gym.closed_loop.reference_host --port 8765` starts the smallest server that passes: a fake model that answers every decision with the same straight-ahead trajectory. Start from it when you wire your own model in.

## Versioning and the schema

The contract is `simforge.model-host/v1`. A change that makes a valid v1 server invalid is a new version; new optional fields are additions to v1. The messages are published as JSON Schema (draft 2020-12), `model-host.v1.schema.json`, packaged with `simforge-gym` beside the conformance command. It has one definition per message: `Health`, `OpenSessionRequest`, `OpenSessionResponse`, `InvokeRequest`, `InvokeResponse` and `Refusal`. Camera JPEGs are `bytes` on the MessagePack wire; the schema describes them as base64 strings only to stay JSON.

## Transport

| Rule | Detail |
|---|---|
| Authentication | Every request carries `Authorization: Bearer <token>`. A host verifies it and answers a wrong or missing one with HTTP 401. |
| Encodings | `/healthz`, session creation and the refusals are JSON. The invoke request is MessagePack (`application/msgpack`). The invoke answer must be `application/msgpack`. |
| One host | A run contacts only the host it was configured with. A failure is never retried and never answered by another model. |
| Unknown fields | A host refuses unknown fields in a session or invoke request, so the names on this page are exact. |

## `GET /healthz`: the declaration

The loop sends the model exactly what the host declares, renders with the settings it declares and adopts its plans after the delays it declares. There is no guessing: v1 requires the declaration, and hosted evaluations require `capabilities.observation` before rendering. Local `simforge-gym` may use a registered family's observation contract and record `observation_contract_undeclared`; an undeclared unknown family is refused with `host_observation_undeclared`.

| Field | Required | Meaning |
|---|---|---|
| `ok` | yes | `true`. |
| `family` | yes | Nonempty name of the model family. |
| `camera_profile` | yes | The rig preset SimForge renders for the model: `alpamayo-4cam` (cameras 0, 1, 2, 6), `alpamayo-6cam` (0, 1, 2, 3, 5, 6) or `qwen-drive-3cam` (0, 1, 2). |
| `horizon_s` | no | Seconds a plan covers (default 6.4). Positive and finite. |
| `samples.session` | no | Trajectories returned per decision (default 1). The loop drives the first. |
| `samples.openLoop` | no | Native samples of open-loop evaluation, the `numTrajSamples` of the model registry (wire parameter `params.num_traj_samples`). |
| `capabilities.observation` | yes | What the loop feeds the model; see below. |
| `capabilities.cameras.required` | no | Camera indices; must equal the profile's. |
| `capabilities.rig.definition` | no | The calibrated rig the model was trained on. When present, the rendered rig must equal it or the run is refused (`model_rig_mismatch`). |

`capabilities.observation`:

| Field | Default | Rule |
|---|---|---|
| `frameHistoryOffsetsS` | none, required | Nonempty list of seconds before the decision, oldest first: each 0 or less, on the 0.1 s grid, strictly increasing, last exactly 0. `[-0.3, -0.2, -0.1, 0]` is four frames per camera at 10 Hz; `[-1.5, -1.0, -0.5, 0]` four frames over 1.5 s. Every camera sends the same instants. |
| `frameEncoding` | `jpeg` | Only `jpeg`. |
| `requiresRoute` | `false` | `true` adds `route_xyz` to every decision. |
| `egoHistoryS` | `1.5` | Seconds of ego history, a multiple of 0.1. The history holds `egoHistoryS / 0.1 + 1` poses (16 by default). |
| `planAdoptDelayS` | `{first: 0, later: 0}` | Both keys, seconds on the 0.1 s grid. `first` must be 0: the first plan steers at once. `later` must be shorter than the replanning interval (0.5 s by default). |
| `renderOverrides` | `{}` | Only `{"lens.distortion": 0}`: render without lens distortion. |

## `POST /v1/sessions`

Request (JSON): `{"camera_ids": [0, 1, 2, 6], "seed": 0}`, the cameras of the declared profile and an integer seed. A host that does not serve those cameras refuses with `camera_set_invalid`.

Answer (JSON): `{"ok": true, "sessionId": "...", "model": {...}, "gpu": {...}}`. `model` carries four nonempty strings, `family`, `revision`, `checkpointDigest` and `quant`: a run can pin them (`model_revision_mismatch` otherwise) and records them as the model's identity. `gpu` is any nonempty object naming the accelerator; SimForge treats it as opaque provenance. `framesPerCamera` and `frameHistoryOffsetsS` are optional and, when present, must agree with `/healthz`.

## `POST /v1/sessions/{id}/invoke`: one decision

The episode asks for a decision every 0.5 s of simulated time, once enough real history exists: the declared frame history and ego history are never padded.

Request (MessagePack):

| Field | Meaning |
|---|---|
| `cameras` | `[{camera_id, jpeg, t_s}]`: JPEG `bytes`. Only frames the host has not been sent yet, oldest first. The host keeps each camera's window of the declared offsets. The first decision sends the whole window; each later one sends the frames new since. |
| `ego_history_xyz`, `ego_history_rot`, `ego_history_t_s` | The last poses in the ego frame at `t_s` (x forward, y left, z up, origin at the footprint centre): `[x, y, z]` points, 3x3 rotations and times relative to `t_s`, 0.1 s apart, ending at 0 with the identity pose. |
| `t_s` | The decision instant, equal to the newest frame's `t_s`. Increases between decisions. |
| `seed`, `params` | The session seed and the run's parameters (`{}` by default). |
| `route_xyz` | With `requiresRoute`: at least 2 `[x, y, z]` points of the remaining route in the ego frame at `t_s`. |

Answer (MessagePack, `application/msgpack`):

| Field | Required | Meaning |
|---|---|---|
| `ok` | yes | `true`. |
| `trajectories` | yes | One or more plans, each a list of at least 2 finite `[x, y]` or `[x, y, z]` waypoints in the ego frame at `t_s`, `dt_s` apart, the first `dt_s` after `t_s`. Every plan covers the same number of waypoints. |
| `dt_s` | no | Must be `0.1`; assumed when absent. |
| `reasoning` | no | A list whose first entry is the model's text; without it the run records `reasoning_missing`. |
| `timings`, `tokens`, `rng_provenance` | no | Kept in the run's record. |

A plan shorter than `horizon_s` is allowed (the check warns): the vehicle follows the plan it is given.

## Timing

- **Deadline.** A request that has not been answered within the deadline fails the run with `model_host_unreachable`. The closed loop's limit is 600 s per request; set `--deadline-ms` to the budget you want to hold yourself to.
- **Simulated time stands still while the model thinks.** An episode runs in offline simulated time: the wall time of an invoke costs the run's duration, not the vehicle's progress. The only simulated delay is the one you declare in `planAdoptDelayS`: a plan decided at `t` steers from `t + later`, and the first plan from `t`.
- **Order.** One decision at a time per session. `t_s` increases; a host may refuse a decision that does not (`input_error`).

## Errors

A refusal is `{"ok": false, "error": {"code": "...", "message": "..."}}`, answered with HTTP 200 on a session call or with a 4xx status; `error.fields` may name the offending request fields. A host answers its own faults with a 5xx and the same body. The loop never retries.

| What happened | Run fails with |
|---|---|
| The host cannot be reached, closes the connection or misses the deadline | `model_host_unreachable` |
| HTTP 401 or 403 | `model_host_unauthorized` |
| An undecodable body, or a body that is not an object | `model_host_protocol_error` |
| `ok: false` or another HTTP error | the host's `error.code` (or `model_host_error`) |
| An invalid declaration | `observation_contract_invalid` |
| A declared rig that differs from the rendered one | `model_rig_mismatch` |
| No trajectories, fewer than 2 waypoints, non-finite values or a `dt_s` other than 0.1 | `model_output_invalid` |

Use stable, specific codes for your own refusals, such as `missing_fields`, `input_error`, `camera_set_invalid`, `frame_window_incomplete` and `session_not_found`: they appear in your run's failure reason.

## Conformance requirements and loop tolerances

The conformance check reports each violation below as `fail`. Some requirements are stricter than the local gym or the loop's decoding tolerance; hosted evaluations already require an observation declaration.

- v1 requires `capabilities.observation` and `camera_profile`. Hosted evaluations require the observation declaration; only local `simforge-gym` may use a registered family's observation contract with a warning. An undeclared unknown family is refused.
- The invoke answer is `application/msgpack`. The loop also decodes a JSON body.
- `trajectories` holds equally long plans, and as many as `samples.session` when declared. The loop reads only the first plan.
- The opened session's `model` has nonempty `family`, `revision`, `checkpointDigest` and `quant`. The loop requires the object and compares only the keys a run pins.

`capabilities.cameras.required` disagreeing with `camera_profile` is a `warn`: the loop never reads it.

## The checks

| Check | What it verifies |
|---|---|
| `health.reachable`, `health.auth` | `/healthz` answers with the bearer; it is refused without one (warning if not). |
| `health.shape`, `health.observation`, `health.horizon`, `health.cameras`, `health.samples`, `health.rig` | The declaration, field by field, with the offending path and value in the message. |
| `session.open`, `session.unknown_fields` | A session opens with the declared cameras and identifies the model; an unknown field is refused. |
| `invoke.accepted`, `invoke.content_type` | Valid decisions are answered `ok`, as `application/msgpack`. |
| `invoke.trajectories`, `invoke.cadence`, `invoke.horizon`, `invoke.samples`, `invoke.reasoning` | The shape of the answer, element by element. |
| `invoke.deadline` | Every answer arrives within `--deadline-ms`. |
| `invoke.refusal`, `invoke.unknown_fields` | A decision without an ego history is refused with a code, never a 5xx or an `ok` answer; an unknown field is refused. |
| `session.close` | `DELETE /v1/sessions/{id}` succeeds. |