Statim answers typed questions about text or JSON: pick one of several labels (choice), rate on a
scale (score), or give a yes/no probability (noul, alias yes_no). It is a native C++20 engine for System-1
decision models and computes all answers in one encoder forward pass, served from one static binary.
It needs no Python, PyTorch, or GPU at runtime. Statim runs Laya checkpoints and implements
the Jev/Laya POST /v1/systemone protocol, so existing clients can switch by changing the base URL.
statim (Latin): immediately, at once.
At a glance
| Measure | Statim Decide Multilingual 0.10.0, trained | Qwen3-8B, zero-shot |
|---|
| 14-category macro accuracy | 0.826 | 0.706 |
| Banking77 | 0.918 | 0.650 |
| Decisions/s, one RTX 3070 | 104 | ≈6 |
- CPU latency: 535 ms/state, multilingual; 1,683 ms/state, English. RTX 3070: 54 ms and 137 ms.
- Reference parity: 240/240 token sequences, with answers within 1e-4 of Laya.
How it was measured
- Accuracy: the same 11,550 held-out items, questions, and options. Statim was trained; both
baselines were zero-shot. mDeBERTa-v3 XNLI scored 0.488 on the 14 categories and 0.224 on
Banking77's 77 intents. Full results
- Speed: Statim Vulkan f32 reached 104 decisions/s (6.2 ms/decision) in batches of 16 on one RTX
3070. Qwen3-8B reached ≈6 decisions/s through Ollama Q4_K_M with 2 parallel requests;
mDeBERTa-v3 XNLI used CUDA f32 and reached 4 decisions/s.
Measurements
- CPU latency: Ryzen 7 5800X, 16 threads, f32, 30 states × 8 questions, and up to 770 tokens.
Performance
- Parity: CI-gated against the official Python package.
Reproduce
These rows use different workloads. They are separate evidence for accuracy, throughput, latency,
and parity, not one combined benchmark.
Against ONNX Runtime on CPU
The comparison uses the same model and the same token ids on a Ryzen 7 5800X: Statim 0.9.2 against
onnxruntime 1.30 (CPU execution provider). Both engines reproduce the PyTorch reference on 240/240
items before any timing. ORT is called from Python, which adds at most 0.02 % to a run.
| Measure | Statim | ORT |
|---|
| Short inputs, f32, 8 threads, ms per state | 257 | 266 |
| 1,024 tokens, f32, 8 threads, ms | 4,399 | 11,421 |
| Short inputs, f32, 16 threads (SMT), ms per state | 457 | 366 |
| First answer after process start, f32 | 0.45 s | 2.06 s |
| Resident memory at start-up, 8-bit | 276 MiB | 579–581 MiB |
| Install | 5.2 MiB executable | 186 MiB of Python packages |
ORT's dynamic int8 is the fastest variant on short inputs, at 168 ms per state on 8 threads, but it
changes 443 of 2,850 held-out decisions and costs 1.7 accuracy points. Statim's q8_0 changes 30
decisions, with no measurable accuracy change.
The comparison does not cover TensorRT or a GPU run. Full protocol and every row
On a public suite: S1Bench
On S1Bench (13 public subsets, 3,880 items, the harness Lev's authors used), the multilingual model
0.10.0 scores 0.657 macro (0.7.0: 0.638), above its untuned Laya base (0.579) and below Lev, a 4B
LLM on a GPU (0.689), and the hosted Jev (0.761). It trails on reading comprehension; over the seven
subsets from sources it never trained on it scores 0.584 (Lev 0.635, Jev 0.766). The consensus of the
English and multilingual models scores 0.673, 1.7 points below Lev, with ECE 0.116 (Lev 0.115,
Jev 0.091). 0.10.0 results;
protocol and the 0.7.0 run.
Quick start
Linux x86-64 with AVX2 (Haswell or newer), CPU. Three steps: download, start, ask.
# 1. Download the engine (3 MB) and the multilingual model (357 MB), and verify both
curl -fLO https://github.com/BEKO2210/statim/releases/download/v0.11.0/statim-0.11.0-linux-x86_64-cpu.tar.gz
curl -fLO https://github.com/BEKO2210/statim/releases/download/v0.11.0/SHA256SUMS
sha256sum -c --ignore-missing SHA256SUMS && tar -xzf statim-0.11.0-linux-x86_64-cpu.tar.gz
curl -fLO https://huggingface.co/Beko2210/statim-decide-multilingual-base/resolve/main/statim-decide-multilingual-base-q8_0.gguf
curl -fL -o SHA256SUMS.model https://huggingface.co/Beko2210/statim-decide-multilingual-base/resolve/main/SHA256SUMS
sha256sum -c --ignore-missing SHA256SUMS.model
# 2. Start the server (it keeps running; it is ready when it logs "listening")
./statim-0.11.0-linux-x86_64-cpu/statim serve -m multilingual=statim-decide-multilingual-base-q8_0.gguf --port 8080
# 3. In a second terminal: one ticket, three typed questions
curl -s localhost:8080/v1/systemone -d '{
"state": {"subject": "Duplicate charge on invoice #4411",
"body": "We were billed twice for March. Please refund the duplicate today or we will cancel."},
"questions": {
"department": {"type": "choice", "instructions": "Which department should handle this request?",
"criteria": {"billing": "invoices, payments, refunds", "technical": "bugs, outages",
"sales": "pricing, new contracts", "other": "everything else"}},
"urgency": {"type": "score", "instructions": "How urgent is this request?",
"criteria": ["not urgent", "soon", "critical deadline or blocking issue"]},
"refund": {"type": "noul", "instructions": "Does the user explicitly request a refund?"}}}'
The answer (statim-decide-multilingual-base, q8_0; excerpt):
{"answers": {
"department": {"type": "choice", "choice": "billing", "probabilities": {"billing": 0.9948, "technical": 0.0011, "sales": 0.0014, "other": 0.0026}},
"urgency": {"type": "score", "score": 1.585, "legend": {"0": "not urgent", "1": "soon", "2": "critical deadline or blocking issue"}, "probabilities": {"0": 0.0617, "1": 0.2916, "2": 0.6467}},
"refund": {"type": "noul", "noul": 0.9224}}}
choice returns the winning label, score the expected level on the legend scale, and noul the
probability that the answer is yes.
The playground is at http://127.0.0.1:8080/. To build from source (any other OS or CPU), use a GPU,
or quantize a model, see docs/BUILD.md. A complete application is in the
ticket-triage example.
How it works
The state and all questions become one token sequence with a mask per option. The encoder and
decision head score every option in one pass. The converter stores architecture, calibration, and
the tokenizer in GGUF, making each model a self-describing artifact.
Why Statim
| Measure | Laya | Statim |
|---|
| Runtime | Python stack | 5.5 MB + .gguf |
| Answers | Reference | 240/240; within 1e-4 |
| Cold start to first answer | 11.35 s | 0.76 s |
| Server memory | 3,915 MB | 650 MB |
| HTTP throughput | 0.43 req/s | 1.07 req/s |
- Runtime: Laya needs Python 3.10+, PyTorch, and transformers. Statim uses one 5.5 MB binary and one
.gguf file.
- Answers: the comparison covers 240/240 token sequences; answers are within 1e-4 of the reference,
which is one step of the 4-decimal rounding both use. CI fails at two steps (
engine_parity_*, on
CPU, Vulkan and CUDA).
- Tokenizer: Laya uses HF
tokenizers (Rust). Statim's native C++ tokenizer is identical on 3,906
cases plus 140k fuzz strings and about 10× faster.
- Memory and throughput: Statim's mmap'd weights are shared between processes. Its 1.07 req/s uses
6 threads; with 4 threads it reaches 0.99 req/s.
- Server: Laya uses FastAPI and runs one inference at a time. Statim has a worker pool, admission
control, bearer auth, Prometheus metrics, JSON logs, health/readiness, request IDs, and graceful
shutdown.
- Deployment: Laya uses pip or Docker. Statim uses a static binary, distroless Docker, and a hardened
systemd unit.
The comparison uses the laptop CPU protocol under CPU performance. Consensus
across the two base checkpoints is reported separately in the research results.
Models
Statim runs any Laya checkpoint. Published Statim Decide models use licence-audited fine-tuning data
and must pass a no-harm promotion gate.
Files: f32 1.58 GB and q8_0 0.45 GB (English); f32 0.91 GB and q8_0 0.36 GB (multilingual).
| Suite | English | Multilingual |
|---|
| typed-decisions | 0.768 | 0.775 |
| Banking77 | 0.928 | 0.918 |
| MASSIVE | 0.867 (en) | 0.816 (12 languages) |
These are trained-suite promotion-gate results. Under the same published protocol,
typed-decisions scores are meraGPT 0.768, laya-typed-decisions 0.766, and Jev 0.727; supervised
MPNet scores 0.941 on Banking77. See REPRODUCE.md.
hf download Beko2210/statim-decide-en-large statim-decide-en-large-q8_0.gguf --local-dir models # pip install huggingface_hub
./statim serve -m english=models/statim-decide-en-large-q8_0.gguf
API
| Endpoint | Purpose |
|---|
POST /v1/systemone | Decide for one state |
POST /v1/systemone/batch | Decide for many states |
GET /v1/models | List loaded models and adapters |
GET /health, GET /ready | Liveness and readiness. |
GET /metrics | Prometheus metrics |
GET / | Browser playground |
The single-state response returns model, answers, usage, and routing. Batch requests use
shared questions and packed forward passes. The playground is enabled by default. Prometheus covers
request, latency, token, concurrency, worker, batch, model, and adapter metrics.
API v1 is frozen as of 0.9.0; contract and real-server tests reject breaking changes in CI.
Requests contain state and questions, with optional model, adapter, lang,
min_confidence, return_logits, calibrate, and ensemble. model may select a loaded model or
consensus; omission enables routing. A positive min_confidence adds escalate to answers below
the threshold. The complete schema, question types, responses, errors, routing, batching, limits,
and operations guide is docs/API.md.
LoRA adapters
One base model can serve small per-category LoRA adapters selected by name or question family.
python tools/convert_lora.py runs/emotion-lora -o models/emotion.lora.gguf \
--base models/laya-multilingual-q8_0.gguf --category emotion # PEFT adapter -> GGUF
statim serve -m multilingual=models/laya-multilingual-q8_0.gguf \
--adapter multilingual:emotion=models/emotion.lora.gguf
curl -s localhost:8080/v1/systemone -d '{"state": "...", "adapter": "emotion", "questions": {...}}'
The converter accepts plain LoRA on encoder attention and MLP projections, rejects unsupported
variants and modules, and binds the adapter to its base by a fingerprint of the checkpoint's norms
and biases, plus, since 0.8.1, a checkpoint SHA-256 over every source tensor; both are shared by a
checkpoint's f32 and quantized files, and the engine refuses an adapter whose base does not match.
- f32, f16, and bf16 default to merging
W + B·A at load. Requests retain base latency; each
adapter owns a copy of the adapted weights, 438 MB for the multilingual f32 model.
- Quantized weights default to runtime LoRA. A rank-4 adapter uses 3.3 MB and adds 17–28 % CPU
latency. Merging the test adapter into q4_0 loses 91 % of its effect.
--adapter-mode merge|runtime overrides this choice. "adapter": "auto" matches the 14 category
keywords in every question ID, then its instructions; mixed or unmatched families use the base.
See docs/API.md#lora-adapters for constraints, routing, measurements,
responses, metrics, and tests. Trained adapters and the evidence for each are in
docs/ADAPTERS.md. On 0.7.0, the PII, emotion and safety adapters pass the gate
(safety in a replication on fresh items) and are published: PII,
emotion, safety.
Authentication, limits, and operations
Set STATIM_API_KEY=key1,key2 or pass --api-key-file FILE. Authentication covers inference,
/metrics, and /v1/models; health, readiness, and the playground remain public. Key sources fail
closed and comparisons use constant-time code. Every key must be 32–4096 printable ASCII characters
without whitespace; generate one with openssl rand -hex 32. A non-loopback server without keys emits
an error and exits before loading models. Pass --allow-unauthenticated to opt in explicitly on a
trusted network; that mode retains the auth_off_on_network warning.
Requests are bounded before inference by body, JSON structure, state, question, option, token,
attention, response, concurrency, queue, and deadline limits. See API limits,
the security report, and the production deployment guide.
Client SDKs
The dependency-free Python and TypeScript clients retry 503 responses with backoff.
Install Python with pip install ./clients/python. For TypeScript, run npx tsc, then import the
package.
from statim import Client
client = Client("http://127.0.0.1:8080", timeout=120)
decision = client.decide(
{"subject": "Duplicate charge on invoice #4411"},
{"refund": {"type": "noul", "instructions": "Does the user explicitly request a refund?"}},
model="multilingual",
)
print(decision.answers["refund"].noul, decision.answers["refund"].confidence)
Both clients provide decide, decide_batch, models, health, and ready; both decision methods
accept an adapter option. Yes/no questions use
wire type noul and return YesNoAnswer. See the Python and
TypeScript guides.
Performance
CPU performance
Measured on an Intel Xeon E3-1505M v5 laptop CPU (4 cores, 8 threads, AVX2, turbo off, 31 GB RAM),
using the f32 multilingual checkpoint and Laya 0.3.20 on PyTorch 2.14. The workload is 30 states × 8
questions from tests/data/golden_inputs.json, up to 20 options and 770 tokens. Each engine uses its
best thread count. Scripts are in bench/.
| Measure | Laya | Statim |
|---|
| In-process mean | 1,186 ms | 1,139 ms |
| In-process p50 | 815 ms | 823 ms |
| HTTP rate, 1 client | 0.43 req/s | 1.07 req/s |
| HTTP p50, 1 client | 1,718 ms | 889 ms |
| HTTP p95, 1 client | 4,994 ms | 1,280 ms |
| HTTP rate, 4 clients | 0.53 req/s | 1.12 req/s |
| HTTP p95, 4 clients | 11,958 ms | 4,425 ms |
| Resident memory | 3,915 MB | 650 MB |
| Cold start | 11.35 s | 0.76 s |
| Peak memory, f32 | 2,667 MB | 585 MB |
| Peak memory, q8_0 | 2,667 MB | 234 MB |
| Runtime footprint | PyTorch ≥ 1.2 GB | 5.5 MB |
Against Laya:
- One client: Laya's 0.43 req/s used 4 threads. Statim reached 1.07 req/s with 6 threads (2.5×)
and 0.99 req/s with 4 threads (2.3×), a 2.3–2.5× range. p95 falls by 74 %.
- Four clients: 0.53 against 1.12 req/s is 2.1× the throughput, with p95 11,958 against 4,425 ms.
- In-process: the mean latency per state is 4 % lower; the p50 is about the same.
- Memory and start: 6× less resident memory, and 15× faster from cold start to first answer.
Peak memory for one shot is 4.6× less in f32 and 11× less in q8_0.
- Footprint: the runtime-footprint row compares PyTorch alone with the Statim binary.
On AVX2 without VNNI, q8_0 makes the file 2.5× (multilingual) to 3.5× (English) smaller and cuts
memory 2.5×, but is slower than f32. Its logits
move by up to ~0.4 and 2 of 240 parity answers change. ARM dotprod/i8mm and AVX-512-VNNI are the
intended int8 CPU targets. Four-bit weights are not recommended for this model family.
GPU performance
GPU build and launch instructions are in docs/BUILD.md.
--device accepts cpu, gpu, vulkan, cuda, or a name such as Vulkan0; STATIM_DEVICE sets
the default. Both f32 checkpoints use about 3.3 GB of VRAM. On an RTX 3070 and Ryzen 7 5800X, f32,
using the 30 × 8 golden workload:
| Path | CPU | RTX 3070 | Gain |
|---|
| Multilingual state | 535 ms | 54 ms | ~10× |
| English state | 1,683 ms | 137 ms | ~12× |
| Multilingual rate1 client | 2.68 req/s | 20.5 req/s | 7.7× |
| Multilingual p501 client | 353 ms | 45 ms | — |
| English rate1 client | 0.91 req/s | 8.1 req/s | 8.9× |
| English p501 client | 1,039 ms | 119 ms | — |
| Multilingual max |Δlogit| | 5.9e-4 | 8.8e-5 | 240/240 argmax |
| English max |Δlogit| | 4.7e-4 | 1.6e-4 | 240/240 argmax |
The state rows are in-process measurements per state. The |Δlogit| rows are against the PyTorch
reference, measured on 2026-10-01 (CPU since 0.9.2's packed GEMM); CI fails above 1e-3 and archives
the values of every green run.
Exact f32 is the default. --gpu-fast or STATIM_GPU_FAST=1 enables f16 GPU math: 24.5 ms per
multilingual state instead of 54 ms, still 240/240 argmax, with logits within ~1e-1 rather than
1e-4. English-model backend results, one HTTP client and 60 requests:
| Mode | CPU | Vulkan | CUDA |
|---|
| f32, exact | 0.84 req/s | 8.40 req/s | 8.35 req/s |
f32, --gpu-fast | | 16.4 req/s | 11.7 req/s |
| q8_0 | | 9.0 req/s | 15.7 req/s |
The multilingual exact-f32 result is Vulkan 21.9 req/s and CUDA 16.2 req/s. Vulkan remains the
default; on NVIDIA, q8_0 with CUDA is the fastest measured large-model path with f32 activations.
./build-vk/statim serve --device vulkan -m english=models/laya-english-f32.gguf \
--batch-window-ms 2 --max-batch 16
With this 2 ms micro-batch window, RTX 3070 English-model throughput changed from 8.47 to 8.30 req/s
with one client, 8.50 to 9.64 with 8 clients, and 8.48 to 10.78 with 16 clients. Compatible requests
must share model, adapter, questions, and inference options. Answers match standalone requests within
1e-4. The default window is 0; leave it disabled on CPU.
Results
Trained Statim versus zero-shot general models
Every system receives the same 11,550 held-out items, questions, and options: 37 language cells with
150 items each across 14 categories, plus the first 2,000 test rows of AG News, DAIR Emotion, and
Banking77. Training-mixture overlaps are removed. Statim 0.10.0 was trained on these categories;
Qwen3-8B and mDeBERTa were zero-shot. This compares out-of-the-box systems, not learning methods.
| Measure | Statim | Qwen3-8B | NLI |
|---|
| Parameters | 307M | 8.2B | 279M |
| 14-category macro accuracy | 0.826 | 0.706 | 0.488 |
| Categories won | 12 | 2 | 0 |
| Banking77 | 0.918 | 0.650 | 0.224 |
| AG News | 0.921 | 0.847 | 0.581 |
| Decisions/s | 104 | ≈6 | 4 |
These are Statim 0.10.0 against zero-shot Qwen3-8B and the zero-shot NLI classifier mDeBERTa-v3-base
XNLI (column NLI), measured on the same RTX 3070 where speed is reported.
Qwen3-8B leads on emotion and fact-check (against 0.7.0 it also led on sentiment, safety and PII). See
docs/BASELINES.md for all cells, limitations, speed, and reproduction commands.
Published model gates
Published models must pass validation, held-out-suite, and pooled-family promotion gates before
replacing their base. The full method and charts are in docs/RESULTS.md.
| Suite | English 0.5.0 | Multilingual 0.10.0 |
|---|
| typed-decisions | 0.768 | 0.775 |
| Banking77 | 0.928 | 0.918 |
| MASSIVE | 0.867 English | 0.816 / 12 languages |
| HWU64 (English only) | 0.833 | — |
| AG News (never trained) | 0.939 | 0.921 |
| Emotion (never trained) | 0.588 | 0.530 |
With paired tests on the same items, English passed 91 suites with 10 significant gains, 0 regressions,
and its zero-shot family within noise. Multilingual 0.10.0 passed against 0.7.0 on 91
suites with 6 significant gains, 85 within noise, and 0 regressions; the decision-category family rose
7.7 points (Holm p < 1e-30), and its 14-category macro accuracy is 0.826 (0.7.0: 0.748, 0.4.0: 0.559).
Category-adapter results and promotion evidence are in docs/ADAPTERS.md.
Security and robustness
Statim applies bearer authentication, bounded parsing and inference budgets, admission control,
deadlines, fixed error responses, and load-time model validation. Malformed GGUF metadata, tensors,
hyperparameters, token IDs, vocabularies, calibration tables, fingerprints, and UTF-8 names are
rejected before serving.
libFuzzer harnesses exercise complete request handling, both tokenizers, and GGUF loading under ASan
and UBSan. CI runs each for 60 seconds per push from a cached corpus. Committed seeds and crash inputs
replay under ctest, and model validation adds 20 malformed-model cases. The HTTP security suite runs
against a live server:
python3 tests/security/test_http.py --binary build/statim --model models/laya-multilingual-f32.gguf
Report vulnerabilities privately as described in SECURITY.md.
docs/SECURITY.md has the findings, fixes, fuzzing campaign and coverage.
Platform support
Only what CI builds and tests, or what was run on real hardware, is listed as supported. Everything
else may build through ggml but is untested.
| Platform | Release binary | CI | Tested on hardware |
|---|
| Linux x86-64, CPU (AVX2, FMA, F16C, BMI2) | yes; glibc 2.28 or newer (0.9.5 and earlier: 2.38) | build, full ctest, ASan + UBSan, fuzzing | Ryzen 7 5800X; Xeon E3-1505M v5 |
| Linux x86-64, Vulkan | yes | builds in the release job only; no GPU runner | RTX 3070, driver 580.159.03: the four *_vulkan parity tests pass (2026-10-01) |
| Linux x86-64, CUDA | no; build from source | no | RTX 3070, CUDA 12.8: the four *_cuda parity tests pass (2026-10-01) |
| x86-64 without AVX2 | no; the release binaries stop with a message naming the missing features | cpu_check test | Core i5-2520M and i3-3227U: that message, not a crash (BUILD.md) |
| Android arm64 (NEON, dotprod) | no; build from source with the NDK (BUILD.md) | cross-build | Galaxy A15 (Dimensity 6100+) and Galaxy Tab S9 Ultra (Snapdragon 8 Gen 2, i8mm): the native test suite passes, multilingual and English parity 240/240 (2026-10-01/02) |
| Linux arm64, macOS, Windows | no | no | not tested |
Status and roadmap
The current release is v0.11.0: Linux x86-64 binaries for CPU and Vulkan, CUDA from source (see
platform support), two published models, client SDKs, and a public demo. 0.9.2 brings a packed f32 GEMM that makes Statim level with or faster than ONNX Runtime on short
inputs; 0.9.1 adds the comparison with ONNX Runtime and a structural GGUF check before
ggml parses a model file; 0.9.0 freezes HTTP API v1, with contract
tests against the real server; 0.8.7 makes the README readable on phones; 0.8.6 gives it a three-step quick start; 0.8.5 publishes the safety adapter after a pre-registered replication; 0.8.4 publishes the PII and emotion adapters; 0.8.3 brings LoRA adapters to both client SDKs; 0.8.2 checks every document against
the code in CI and records the first category-adapter experiment; 0.8.1 binds each LoRA adapter to the
exact checkpoint it was trained on; 0.8.0 added per-category LoRA adapters,
the comparison with a local LLM and an NLI classifier, fuzzing in CI, load-time model and adapter
validation, and a warning when a non-loopback server starts without authentication. Three
category adapters are published: PII, emotion and safety (docs/ADAPTERS.md).
Next, sentiment and fact-check need more data or larger held-out samples.
Since 0.9.0, API v1 accepts only additive changes; breaking changes require a new major version.
docs/COMPATIBILITY.md states what each version promises, the deprecation policy,
and how to upgrade and roll back.
See the changelog and roadmap.
Licence
Weights are meant to use only commercially usable, non-ShareAlike data; sources are in
DATA_LICENSES.md. A re-audit on 2026-10-03 found that the weights released so far
break that rule, so they are offered for noncommercial use only until cleared replacements ship
(licence findings). Original Laya checkpoints are Apache-2.0. Statim is independent
and not affiliated with the Laya authors or TypeSafe; see NOTICE.
