// Command reference
simforge bench
Generated from the CLI's own help (CLI 0.2.0-rc.5). Global flags and exit codes are on the reference index.
simforge bench
Benchmark the renderer on this machine: the Helios Score, per-factor sub-scores, frame correctness
simforge bench [OPTIONS]
simforge bench <COMMAND>Runs on: local runtime
| Argument / flag | Notes | What |
|---|---|---|
| --suite SUITE | one of quick, full; default quick | Which scenes to run |
| --scenes ID | repeatable | Only these scene ids (comma-separated), from the suite's pack |
| --set KEY=VALUE | repeatable | Diagnostics: a render-config override (simforge render --set keys) on every scene. It changes the workload, so the result is recorded with its overrides and has no score (status modified) |
| --gpu N | The GPU to benchmark: its index in nvidia-smi (NVIDIA) or in the adapter list simforge doctor prints. Default: the renderer's choice (the first discrete GPU) | |
| --out FILE | Write the result JSON here too (default: simforge-bench-<time>.json in the current directory) | |
| --pack DIR | A local pack directory instead of the pinned pack (maintainers; also SIMFORGE_BENCH_PACK) | |
| --keep DIR | Keep every sampled frame and the session logs in this directory | |
| --download-only | Download and verify the suite's inputs (pack, map tier, actors, sky), then exit without rendering | |
| --allow-busy | Run even when another process is using the GPU (the result is flagged gpu-shared) | |
| --stall-timeout-s SECONDS | default 300 | Kill a scene session that makes no progress for this long (a hung GPU submission); the scene is reported failed and the remaining scenes run in a new session |
| --quiet | No progress or table on stderr (the JSON result on stdout is unchanged) | |
| --cache-root DIR | Map cache root. Default: SIMFORGE_MAPS_CACHE_ROOT, then $XDG_DATA_HOME/simforge/maps | |
| --assets-root DIR | Actor-asset root. Default: SIMFORGE_ACTOR_ASSETS_ROOT, then $XDG_DATA_HOME/simforge/actor-assets |
simforge bench compare
Compare two results: per-scene time deltas against the noise of both runs, and frame identity
simforge bench compare [OPTIONS] <A.json> <B.json>
Runs on: local runtime
| Argument / flag | Notes | What |
|---|---|---|
| <a.json> | required | The baseline result |
| <b.json> | required | The candidate result |
| --suite SUITE | one of quick, full; default quick | Which scenes to run |
| --scenes ID | repeatable | Only these scene ids (comma-separated), from the suite's pack |
| --set KEY=VALUE | repeatable | Diagnostics: a render-config override (simforge render --set keys) on every scene. It changes the workload, so the result is recorded with its overrides and has no score (status modified) |
| --gpu N | The GPU to benchmark: its index in nvidia-smi (NVIDIA) or in the adapter list simforge doctor prints. Default: the renderer's choice (the first discrete GPU) | |
| --out FILE | Write the result JSON here too (default: simforge-bench-<time>.json in the current directory) | |
| --pack DIR | A local pack directory instead of the pinned pack (maintainers; also SIMFORGE_BENCH_PACK) | |
| --keep DIR | Keep every sampled frame and the session logs in this directory | |
| --download-only | Download and verify the suite's inputs (pack, map tier, actors, sky), then exit without rendering | |
| --allow-busy | Run even when another process is using the GPU (the result is flagged gpu-shared) | |
| --stall-timeout-s SECONDS | default 300 | Kill a scene session that makes no progress for this long (a hung GPU submission); the scene is reported failed and the remaining scenes run in a new session |
| --quiet | No progress or table on stderr (the JSON result on stdout is unchanged) | |
| --cache-root DIR | Map cache root. Default: SIMFORGE_MAPS_CACHE_ROOT, then $XDG_DATA_HOME/simforge/maps | |
| --assets-root DIR | Actor-asset root. Default: SIMFORGE_ACTOR_ASSETS_ROOT, then $XDG_DATA_HOME/simforge/actor-assets |
simforge bench pack actors
Cut the pack's actor closure (only its scenes' catalog entries and models, licences carried) from the SDK's closure
simforge bench pack actors [OPTIONS] --out <DIR> <PACK_DIR>
Runs on: local runtime
| Argument / flag | Notes | What |
|---|---|---|
| <pack_dir> | required | The pack directory (its scenarios' traces name the catalog ids) |
| --suite SUITE | one of quick, full; default quick | Which scenes to run |
| --scenes ID | repeatable | Only these scene ids (comma-separated), from the suite's pack |
| --set KEY=VALUE | repeatable | Diagnostics: a render-config override (simforge render --set keys) on every scene. It changes the workload, so the result is recorded with its overrides and has no score (status modified) |
| --gpu N | The GPU to benchmark: its index in nvidia-smi (NVIDIA) or in the adapter list simforge doctor prints. Default: the renderer's choice (the first discrete GPU) | |
| --out FILE | Write the result JSON here too (default: simforge-bench-<time>.json in the current directory) | |
| --pack DIR | A local pack directory instead of the pinned pack (maintainers; also SIMFORGE_BENCH_PACK) | |
| --keep DIR | Keep every sampled frame and the session logs in this directory | |
| --download-only | Download and verify the suite's inputs (pack, map tier, actors, sky), then exit without rendering | |
| --allow-busy | Run even when another process is using the GPU (the result is flagged gpu-shared) | |
| --stall-timeout-s SECONDS | default 300 | Kill a scene session that makes no progress for this long (a hung GPU submission); the scene is reported failed and the remaining scenes run in a new session |
| --quiet | No progress or table on stderr (the JSON result on stdout is unchanged) | |
| --cache-root DIR | Map cache root. Default: SIMFORGE_MAPS_CACHE_ROOT, then $XDG_DATA_HOME/simforge/maps | |
| --assets-root DIR | Actor-asset root. Default: SIMFORGE_ACTOR_ASSETS_ROOT, then $XDG_DATA_HOME/simforge/actor-assets | |
| --out DIR | required | Write the closure root (closures/<digest>.json, blobs/sha256/..) here, ready to publish |
| --base SHA256:BYTES | The closure to cut from (sha256 and byte size of its document). Default: the SDK's pinned actor closure |
simforge bench pack record
Record the reference (timings, frame hashes, sample frames) into a pack directory on this machine
simforge bench pack record [OPTIONS] --label <LABEL> <PACK_DIR>
Runs on: local runtime
| Argument / flag | Notes | What |
|---|---|---|
| <pack_dir> | required | The pack directory (pack.json, scenarios, rigs); reference/ is (re)written |
| --suite SUITE | one of quick, full; default quick | Which scenes to run |
| --scenes ID | repeatable | Only these scene ids (comma-separated), from the suite's pack |
| --set KEY=VALUE | repeatable | Diagnostics: a render-config override (simforge render --set keys) on every scene. It changes the workload, so the result is recorded with its overrides and has no score (status modified) |
| --gpu N | The GPU to benchmark: its index in nvidia-smi (NVIDIA) or in the adapter list simforge doctor prints. Default: the renderer's choice (the first discrete GPU) | |
| --out FILE | Write the result JSON here too (default: simforge-bench-<time>.json in the current directory) | |
| --pack DIR | A local pack directory instead of the pinned pack (maintainers; also SIMFORGE_BENCH_PACK) | |
| --keep DIR | Keep every sampled frame and the session logs in this directory | |
| --download-only | Download and verify the suite's inputs (pack, map tier, actors, sky), then exit without rendering | |
| --allow-busy | Run even when another process is using the GPU (the result is flagged gpu-shared) | |
| --stall-timeout-s SECONDS | default 300 | Kill a scene session that makes no progress for this long (a hung GPU submission); the scene is reported failed and the remaining scenes run in a new session |
| --quiet | No progress or table on stderr (the JSON result on stdout is unchanged) | |
| --cache-root DIR | Map cache root. Default: SIMFORGE_MAPS_CACHE_ROOT, then $XDG_DATA_HOME/simforge/maps | |
| --assets-root DIR | Actor-asset root. Default: SIMFORGE_ACTOR_ASSETS_ROOT, then $XDG_DATA_HOME/simforge/actor-assets | |
| --label LABEL | required | A label for the reference machine (e.g. "RTX 3080 desktop") |
| --gpu N | ||
| --cache-root DIR | ||
| --assets-root DIR | ||
| --from-result RESULT.json | Record from this result of a run kept in <PACK_DIR>/reference/.record instead of running | |
| --scenes ID | repeatable | Record only these scenes (comma-separated); the others keep their reference |
simforge bench pack seal
Seal a pack directory as a content-addressed closure (writes the closure document and prints its pin)
simforge bench pack seal [OPTIONS] --out <DIR> <PACK_DIR>
Runs on: local runtime
| Argument / flag | Notes | What |
|---|---|---|
| <pack_dir> | required | The pack directory |
| --suite SUITE | one of quick, full; default quick | Which scenes to run |
| --scenes ID | repeatable | Only these scene ids (comma-separated), from the suite's pack |
| --set KEY=VALUE | repeatable | Diagnostics: a render-config override (simforge render --set keys) on every scene. It changes the workload, so the result is recorded with its overrides and has no score (status modified) |
| --gpu N | The GPU to benchmark: its index in nvidia-smi (NVIDIA) or in the adapter list simforge doctor prints. Default: the renderer's choice (the first discrete GPU) | |
| --out FILE | Write the result JSON here too (default: simforge-bench-<time>.json in the current directory) | |
| --pack DIR | A local pack directory instead of the pinned pack (maintainers; also SIMFORGE_BENCH_PACK) | |
| --keep DIR | Keep every sampled frame and the session logs in this directory | |
| --download-only | Download and verify the suite's inputs (pack, map tier, actors, sky), then exit without rendering | |
| --allow-busy | Run even when another process is using the GPU (the result is flagged gpu-shared) | |
| --stall-timeout-s SECONDS | default 300 | Kill a scene session that makes no progress for this long (a hung GPU submission); the scene is reported failed and the remaining scenes run in a new session |
| --quiet | No progress or table on stderr (the JSON result on stdout is unchanged) | |
| --cache-root DIR | Map cache root. Default: SIMFORGE_MAPS_CACHE_ROOT, then $XDG_DATA_HOME/simforge/maps | |
| --assets-root DIR | Actor-asset root. Default: SIMFORGE_ACTOR_ASSETS_ROOT, then $XDG_DATA_HOME/simforge/actor-assets | |
| --out DIR | required | Write the closure root (closures/<digest>.json, blobs/sha256/..) here, ready to publish |
simforge bench run
Run the benchmark (the default when no subcommand is given)
simforge bench run [OPTIONS]
Runs on: local runtime
| Argument / flag | Notes | What |
|---|---|---|
| --suite SUITE | one of quick, full; default quick | Which scenes to run |
| --scenes ID | repeatable | Only these scene ids (comma-separated), from the suite's pack |
| --set KEY=VALUE | repeatable | Diagnostics: a render-config override (simforge render --set keys) on every scene. It changes the workload, so the result is recorded with its overrides and has no score (status modified) |
| --gpu N | The GPU to benchmark: its index in nvidia-smi (NVIDIA) or in the adapter list simforge doctor prints. Default: the renderer's choice (the first discrete GPU) | |
| --out FILE | Write the result JSON here too (default: simforge-bench-<time>.json in the current directory) | |
| --pack DIR | A local pack directory instead of the pinned pack (maintainers; also SIMFORGE_BENCH_PACK) | |
| --keep DIR | Keep every sampled frame and the session logs in this directory | |
| --download-only | Download and verify the suite's inputs (pack, map tier, actors, sky), then exit without rendering | |
| --allow-busy | Run even when another process is using the GPU (the result is flagged gpu-shared) | |
| --stall-timeout-s SECONDS | default 300 | Kill a scene session that makes no progress for this long (a hung GPU submission); the scene is reported failed and the remaining scenes run in a new session |
| --quiet | No progress or table on stderr (the JSON result on stdout is unchanged) | |
| --cache-root DIR | Map cache root. Default: SIMFORGE_MAPS_CACHE_ROOT, then $XDG_DATA_HOME/simforge/maps | |
| --assets-root DIR | Actor-asset root. Default: SIMFORGE_ACTOR_ASSETS_ROOT, then $XDG_DATA_HOME/simforge/actor-assets | |
| --suite SUITE | one of quick, full; default quick | Which scenes to run |
| --scenes ID | repeatable | Only these scene ids (comma-separated), from the suite's pack |
| --set KEY=VALUE | repeatable | Diagnostics: a render-config override (simforge render --set keys) on every scene. It changes the workload, so the result is recorded with its overrides and has no score (status modified) |
| --gpu N | The GPU to benchmark: its index in nvidia-smi (NVIDIA) or in the adapter list simforge doctor prints. Default: the renderer's choice (the first discrete GPU) | |
| --out FILE | Write the result JSON here too (default: simforge-bench-<time>.json in the current directory) | |
| --pack DIR | A local pack directory instead of the pinned pack (maintainers; also SIMFORGE_BENCH_PACK) | |
| --keep DIR | Keep every sampled frame and the session logs in this directory | |
| --download-only | Download and verify the suite's inputs (pack, map tier, actors, sky), then exit without rendering | |
| --allow-busy | Run even when another process is using the GPU (the result is flagged gpu-shared) | |
| --stall-timeout-s SECONDS | default 300 | Kill a scene session that makes no progress for this long (a hung GPU submission); the scene is reported failed and the remaining scenes run in a new session |
| --quiet | No progress or table on stderr (the JSON result on stdout is unchanged) | |
| --cache-root DIR | Map cache root. Default: SIMFORGE_MAPS_CACHE_ROOT, then $XDG_DATA_HOME/simforge/maps | |
| --assets-root DIR | Actor-asset root. Default: SIMFORGE_ACTOR_ASSETS_ROOT, then $XDG_DATA_HOME/simforge/actor-assets |