# Train with SimForge Gym

> Train driving policies with SimForge Gym (the open simforge-gym package) on episodes served by simforge env serve, on your own machine. Reinforcement learning today; supervised fine-tuning recipes are coming.

Source: https://simforge.ai/docs/gym

`simforge env serve` runs episodes of a scenario on the local runtime, with reactive traffic and optional rendered cameras. SimForge Gym is where you train driving models on SimForge: reinforcement learning today, supervised fine-tuning recipes coming. Its open `simforge-gym` Python package (formerly `simforge-rl`) is the Gymnasium client: it never contains a simulator of its own, so without a running server there is no episode. Training runs on your machine; the cloud does not run it.

Install it from PyPI:

```sh
python -m pip install 'simforge-gym[ppo]'
```

## Serve episodes

```sh
simforge env serve s.json --socket /tmp/simforge-env.sock --no-sensors --max-sessions 4
```

The server prints a ready document and runs until you stop it. `--no-sensors` serves state only and needs no GPU. For camera observations, serve with `--rig rig.json` instead (see [Preview and render](/docs/render) for rigs). `--max-sessions` must cover every environment you open, including the evaluation one.

On Windows use a runtime advertising `capabilities.envSocketTransport: "named-pipe"`. The same `--socket` path selects a secure local named pipe with a bounded hashed name, so long profile paths do not hit the Unix socket path limit. Older Windows runtimes fail with an upgrade instruction. macOS and Linux use Unix sockets.

## Drive it from Python

```python
from simforge_gym import SimForgeEnv, SocketTransport, WeightedTerms

with SimForgeEnv(
    SocketTransport("/tmp/simforge-env.sock"),
    action_mode="setpoint",
    observation="state",
    reward_fn=WeightedTerms({"progress": 1.0, "proximity": 2.0}),
) as env:
    observation, info = env.reset(seed=7)
    observation, reward, terminated, truncated, info = env.step([5.0, 1.0])
```

- A `setpoint` action is `[target_speed_mps, target_accel_mps2]`; `action_mode="control"` takes `[throttle, brake, steer]`; `None` keeps the scenario's scripted control.
- `observation="state"` is privileged engine state. `"sensors"` returns only rendered camera passes keyed `<sourceId>/<pass>`, with no engine state in `info`; `"state+sensors"` returns both.
- `reward_fn` replaces the reward; `info["native_reward"]` keeps the runtime's own.
- Reset after an episode terminates or truncates; nothing resets on its own.

## Train with PPO

```sh
python -m simforge_gym.examples.ppo --tasks richmond-field-station --runtime --num-envs 8 --total-timesteps 200000 --out run/
```

State observations need no GPU. `--tasks` picks a bundled task set: one per released map (`belmont-research-center`, `easterbrook-discovery-school`, `el-camino-road`, `richmond-field-station`, `yale-street`), each 24 to 29 weighted, seeded scenario templates (car following, cut-ins, lead braking, crossing pedestrians, unprotected turns, red-light runners) at sites verified on that map. A set is pinned to the map version it was verified on: pull that version. `--runtime` runs each worker's episodes in its own runtime process; without it the example starts `simforge env serve` processes instead, with the same observations and rewards. `run/` then holds `curve.csv`, `curve.svg`, `metadata.json` and `policy.pt`, and each evaluation also scores the scenarios' scripted driving as a baseline. `python -m simforge_gym.examples.ppo --help` lists every flag.

## Train on a pulled dataset

```sh
simforge datasets pull <dataset-id> --include scenarios --out data/my-dataset
python -m simforge_gym.examples.ppo --dataset data/my-dataset --split train --runtime --max-open 16 --eval-limit 8 --out run/
```

`--dataset` takes a pulled folder or a hosted dataset id, which it pulls first (see [Getting data out](/docs/datasets)); every file is checked against its digest, and each episode runs exactly the input its scenario's simulation ran, on the installed map the scenario names. `--max-open` bounds the sessions each worker keeps open, so a split of thousands of scenarios trains with a bounded number of sessions. In Python, `ScenarioSet.from_dataset(...)` and `ScenarioSet.bundled(...)` give the same task sets.

## Evaluate a driving model in closed loop

`simforge_gym.closed_loop.episode` drives one episode of an imported scenario with a camera-based driving model, rendering its cameras live on your GPU with `simforge env serve`. It is the runner SimForge cloud evaluation uses, so the episode folder is scored the same way:

```sh
python -m simforge_gym.closed_loop.episode --plan plan.json --out episode/
python -m simforge_gym.eval.score episode/
```

The plan names the scenario workspace, its camera ego, the rig and the model: a model host over HTTP (`modelHost`) or a checkpoint loaded in this process (`localPolicy`, which needs `simforge-gym[train]`). The score reports driving score, route completion, collisions, infractions and comfort. The plan fields are in the package README.

## Persistent worlds

`simforge world serve` hosts one shared, realtime world that several clients join and drive, rather than isolated episodes. It authenticates clients with a token from a file you create first and keep private (for example `umask 077; head -c 32 /dev/urandom | xxd -p > world.token`), passed as `--token-file`; see `simforge world serve --help`.