Sandbox lifecycle · Key features · Latency · Quick start · Status
One machine, thousands of sandboxes. Sandstorm pauses idle sandboxes and snapshots them to disk, then restores them quickly when a request arrives, so one machine can hold far more sandboxes than it could run at once. It supports two sandbox backends, gVisor containers and Firecracker microVMs. Every sandbox has its own network namespace, and both backends can run in one fleet, each forming a resource pool of its own. The master serves a web console for viewing and managing sandboxes and scheduling across the fleet.
The 64 MiB app, p50 of 20 started one at a time, log scales. Each panel is ranked on its own, best first, in gold; Sandstorm's other rows are in blue. FC: Firecracker. Cube: CubeSandbox. Also shown: microsandbox, Kata, Docker, Podman, bubblewrap. Warm: restored from an image of the app already started. Cloud VM: six 2-core, 2 GB nodes for gVisor and containers; one 2-core, 3.6 GB VM under PVM for Firecracker and CubeSandbox. KVM machine: 2 × Xeon E5-2637 v2, 31 GB. Clones are of a running app, until all 20 answer (microsandbox: until running).
Adaptive Sandbox Lifecycle
Agents spend much of their time waiting. Sandstorm is designed around this behavior, with four sandbox tiers that progressively suspend idle sandboxes and quickly wake them when requests arrive.
| Tier | State | Memory | Back to hot |
|---|
| hot | Running | Held | — |
| frozen | Paused, cold pages in zswap | Partly held | ~0.1 s |
| snapshot | Checkpointed to disk | Freed | ~0.2 s |
| detached | Only the volume remains | Freed | Cold start |
Key Features
-
One master, many nodes. The master keeps desired state in SQLite and places sandboxes. Nodes join by address and pairing code, or dial out from behind NAT.
-
Wake on traffic. A gateway on the sandbox's node wakes it and forwards the request.
-
Node self-protection. Nodes refuse new work when memory runs low, so SSH stays reachable.
-
One binary. sandstormd serves as the master, node, and web console.
-
Tuned for each backend. gVisor and Firecracker each get their own optimizations.
The master's web console shows sandbox states, tier transitions, and node headroom.
Latency
Measured 2026-10-31, with requests from each node to its own gateway, so no network is in the path. gVisor needs no KVM and ran on six cloud nodes. Firecracker ran on one machine with KVM and one cloud VM under PVM. The hosts differ, so read each column on its own.
| gVisor | Firecracker, KVM | Firecracker, PVM |
|---|
| Host | 2 cores, 2 GB | 16 threads, 31 GB | 2 vCPU, 3.8 GB |
| Wake from frozen, 64 MiB app | 100 ms | 165 ms | 78 ms |
| Wake from disk, 16 / 64 / 256 MiB app | 130 / 150 / 720 ms | 150 / 150 / 500 ms | 80 / 135 / 450 ms |
| Wake from disk, 512 MiB / 1 GiB app | 1.1 / 3.3 s | 0.7 / 1.4 s | 0.4 s / not run |
| Kept in memory per node: web / 64 MiB / 256 MiB app | 76 / 18 / 6 | 293 / 160 / 85 | 45 / 34 / 18 |
| Refused at 1, 2, 4, 8 requests a second per node | 2.1 / 0.2 / 0.1 / 1.6% | 0.2 / 0 / 0 / 0% | 0 / 0 / 0 / 0% |
| Answered, of 10 / 40 woken at the same instant | 6 / 6 | 10 / 28 | 5.5 of 8 |
Traffic is 40 sandboxes per node with a few getting most requests; about 12 fit awake on a gVisor node. p50 is 6-10 ms on gVisor, 7-50 ms on KVM and 4-14 ms on PVM; p99 is about 2 s, 0.8-1.9 s and 1.8-3.6 s, the requests that had to wait for a restore. A wake that cannot start within 2 s is refused rather than queued. A gVisor node here restores two at a time and queues four, so of 10 or more woken at once, six are answered and the rest get a 503 at once.
The "kept in memory" metric counts sandboxes that remained running with swap enabled; it does not indicate how quickly they can respond to requests.
Without swap, the PVM host could accommodate only about eight instances of the 64 MiB app. As a result, the traffic test used a smaller fleet of 20 sandboxes, and the 1 GiB app was not tested.
Firecracker's results for the 512 MiB and 1 GiB configurations are based on a single sample each.
See the full results and testing methodology.
Method
The figure at the top: every runtime started one at a time, polled every 5 ms, p50 of 20 (compare.py). Sandstorm's time includes the master placing the sandbox and the node setting up its network. The cloud column is the median of six 2-core, 2 GB nodes, and for Firecracker and CubeSandbox one 2-core, 3.6 GB VM with CubeSandbox's PVM host kernel, which gives a VM without KVM a /dev/kvm.
A warm template restores an image of the app already started, as CubeSandbox always does; it is opt-in here, because every sandbox starts from the same memory and the template has to say how to reseed. CubeSandbox's API always restores a snapshot, so its cold figure comes from its own cubemastercli multirun, with a template's create request minus the snapshot annotations and a 5 ms probe: from the client's call until the program answers, as for the rest.
This is the beginning of the README. Read the rest at the original. Read the full README ↗
