interlock: local AI device router

Version 1.0.11: a research instrument. From 1.0, versions follow semantic versioning: a change that breaks the CLI, the receipt format or the experiment spec format comes with a new major version. interlock is advisory only: it suggests a model, engine and device, and it never serves traffic. Every local number in it was measured on one rig (Intel Core Ultra 9 285K with its NPU and iGPU, plus an RTX 5090), and remote colab:<card> rows live in their own separate evidence pool with full card identity (provider, card, vram_mib, driver_version, cuda_version, pool). Treat its routes as measured evidence for that specific hardware pool, not a general benchmark. Correction in 1.0.6: the time-to-first-token and the concurrency-above-1 throughput that the Linux CUDA generate ladder recorded in 1.0.5 and earlier were not valid measures (each timed prompt was served from Ollama's prompt cache, and concurrent requests queued through one slot); interlock no longer uses them, and version 3 ladder rows replace them.

Railway interlocking stops a train being routed into a conflicting path. This tool does that for devices: it sends each job to a model, an engine and a device.

You give it a task (embed, rerank, NLI, verify, generate) and the constraints that matter (quality, latency, context, thinking level). It answers with a model, an engine, a device and the settings, and it shows the receipt behind every number. OpenVINO's AUTO and HETERO plugins can pick a device once you have already picked the model. Nothing in that stack picks the model. That gap is the product.

Install

From a clone:

uv sync
uv run interlock --help

From PyPI:

pip install mcptoolshop-interlock

uv tool install mcptoolshop-interlock

The distribution name is mcptoolshop-interlock. The import and the command are interlock. Python 3.13, 3.14 and 3.15 (requires-python is >=3.13; 3.13 is what Colab's default image runs). CI runs the tests on Ubuntu on all three, with 3.15 as the main leg. On 3.15 PyYAML installs from source until it publishes cp315 wheels (it builds a pure-Python wheel when libyaml is absent). The intel extra needs Python 3.13 or 3.14: OpenVINO 2026.4.1 has no cp315 wheels yet, so on 3.15 the extra installs transformers but no openvino* (an environment marker); the core works on 3.15. A live measurement run on the rig is written for the Windows rig this table was measured on. The Linux CUDA path (new in 1.0.0) runs on a remote Colab VM.

What this program is allowed to touch is in the threat model.

The receipt schema, the routing table, and the harness came first. interlock route reads that table. It does not spend device time.

What's new in 1.0.0

1.0.0 follows 0.2.0. There was no separate 0.3.0 release; its changes ship in 1.0.0.

• Remote Colab cards as devices. colab:A100-40, colab:A100-80, colab:L4, colab:T4 and colab:G4 rows carry the card's identity (provider, card, vram_mib, driver and CUDA versions, and a zone pool) and sit in their own pool, apart from the rig's rows. See Devices.
• A Linux CUDA measuring path for those cards (interlock run with a platform: linux-cuda spec), with three pre-registered specs: experiments/c2-fill-card/, experiments/c2-generate/ and experiments/c2-throughput/. Failed shapes (oom, load_refused, timeout, ctx_exceeded) are kept as capability rows. A cell that was not measured (generation_aborted, a generation Ollama aborted, or unmeasured, an unrecognised error or a prompt that could not be brought to its length) is kept too, but says nothing about what the card can do. Each generate-ladder row records settings.ladder_version and the prompt length the model actually evaluated.
• Every C2 model names a checked source: an Ollama tag with its registry manifest digest, or a GGUF file with its Hugging Face repo, commit and sha256. Gemma 4 31B uses the Ollama library's Q4_K_M, Q8_0 and BF16 builds; it has no published Q6_K build, so that cell is pre-registered as load_refused. interlock c2-sources <spec> prints each entry's source and pull command, and --verify-live checks every source, digest and quant label against registry.ollama.ai and huggingface.co.
• Throughput inputs are exact token lengths. Each throughput model gets fixed 128-token and 510-token inputs built with its own tokenizer at a pinned revision (510 leaves room for the special tokens inside a 512-token limit), committed in experiments/c2-throughput/inputs.json. Rerank and NLI are out of scope on C2: stock Ollama serves neither, so those cells are pre-registered as load_refused with that reason.
• Opt-in remote routing. interlock route ignores remote rows unless the request passes --allow-remote or --device colab:<card>. A remote winner is marked remote: true with its . See .

The table holds no remote rows yet: no remote window has been imported. See CHANGELOG.md for the full list.

What the table says today

The routing table holds 509 rows from committed receipts, all measured on the rig. A few of the answers it already gives:

• A small model runs well on the Intel side. Phi-3.5-mini decodes at about 20–22 tokens/s on the CPU and the iGPU, and about 15 tokens/s on the NPU, at a 1024-token prompt with thinking off. That leaves the 5090 free for bigger work.
• An 8B model doesn't belong on the Intel side. At the same 1024-token prompt with thinking off, Qwen3-8B decodes at about 10–12 tokens/s on the CPU, iGPU and NPU. On the 5090 it decodes at about 208 tokens/s with thinking off and 211 with it on. Ask for 15 tokens/s with the 5090 busy, and the router refuses and names the experiment behind the gap: interlock route --task generate --model qwen3-8b --min-tok-s 15 --busy 5090.
• Some shapes don't run at all. The NPU rejected Qwen3-Embedding at 512-token inputs, both one at a time and in batches of 8. Those rows are kept as capability findings, never ranked as slow candidates.

Those figures are the decode rates recorded in the E3 and e3-ref receipts. The run notes are in experiments/e3/run-notes-2026-10-10-11.md. interlock route ranks by the worst repeat, not the median, so its choices are more cautious than these summaries.

What it will not do

interlock does not serve traffic and it does not proxy requests. It does not train models. It does not edit offrig, npu-serve or rnd. Those stay with their owners. A serving layer needs its own brief.

Devices

Every device is named.

Name	What it is
NPU	Intel AI Boost on the Core Ultra 9 285K
iGPU	The Intel GPU. A live enumeration must say vendor 0x8086 and device type INTEGRATED. The full name is recorded, not the decision
CPU	The same processor's CPU
5090	The RTX 5090, reached only through Ollama at 127.0.0.1:11434
colab:<card>	Remote Colab CUDA cards (A100-40, A100-80, L4, T4, G4), measured on Linux via NVML or nvidia-smi

GPU.1 is the 5090 as OpenVINO sees it on this rig. The resolver refuses that index even if the name or the vendor changes, refuses a bare GPU, refuses AUTO, HETERO and MULTI, and refuses a GPU that is not vendor 0x8086 and INTEGRATED. A list with neither device property still requires the full name to contain "Intel". A missing device is an error. interlock does not fall off onto a weaker one.

Remote rows carry remote_device identity (provider: "colab", card, vram_mib, driver_version, cuda_version, and a zone pool string—never an account id). Remote rows are kept apart from local rig rows and are never pooled into the rig pool. interlock table --device colab:<card> (or colab:*) lists remote rows, and interlock route considers remote cards only when opted in (--allow-remote or --device colab:<card>). Failed remote shapes (oom, load_refused, timeout, ctx_exceeded) are recorded as capability rows with their measured reason rather than skipped. interlock route ranks generate-ladder rows only within the newest ladder_version present, and its notes say how many older rows it did not compare.

The NPU is exclusive. A 5090 step runs only inside a Publisher grant. The runner records that grant. CPU-heavy legs wait for a window the Publisher announces.

Receipts

A routing row names the task, the model (repo or tag, revision or digest, quantisation, licence), the engine and its version, the device, the settings, and the quality and latency it measured. Quality metrics carry their own n, interval and dataset hash. Latency keeps cold and warm apart. A single run is marked provisional. The row points at the source receipt and the commit it was imported from, or at an experiment spec and a run id.

A run can also record a capability row instead of a speed. device_lost and hang carry no timing. native_binary_rejected — the device compiled the model and refused the native binary at execute — keeps only the compile time as evidence, and is scoped to the model, device and static shape: only arms with that identical key are marked not run, and the device's other shapes still run. All three are provisional and never count as speeds. An arm that an abort stopped mid-schedule is kept in its receipt marked aborted: true, with the slots actually completed; the routing table refuses aborted rows, and table/gaps.md lists the aborted run. A watcher thread guards every run from outside the main thread: a native compile that runs past compile_ceiling_s (default 900 s) is recorded as a compile_timeout capability row, and system RAM at 88% aborts the run; both flush the receipt first and end the process with exit code 75 (compile timeout) or 76 (RAM stop). Every run also writes receipts/<run_id>.run.json before it touches a device (git-ignored, not a receipt), so a run killed from outside still leaves a record that it started. To stop a run cleanly from outside, create receipts/<run_id>.stop: the watcher flushes the receipt and exits with code 77.

interlock check

That validates every receipt under receipts/ and table/routing.jsonl. An empty tree is fine. A home-directory path in a receipt is not.

Run lifecycle

A run ends in one of two states. A finished run leaves receipts/<run-id>.jsonl and, for E4b, its series beside it. A run that stops partway writes that receipt first, with the calls that finished and aborted: true, then moves the receipt, the series, the log and the run marker into aborted/<run-id>-<sha>/ with a NOTE.json (the run id, the harness sha, the start and end, the exception, and the files). aborted/ is gitignored. It is not a result: interlock import does not read it, and those runs never reach the routing table.

receipts/ must not hold a *-series/ directory that has no receipt. The next run refuses to start while one is there, and names interlock runs quarantine <run-id>.

interlock runs list
interlock runs quarantine <run-id>

list prints each run id and a state: complete, aborted, or stray. quarantine moves a stray into aborted/ with a note that it was set aside by hand. Neither command starts Ollama or a device. --quarantine chooses the set-aside directory; the default is aborted/ under the run root.

With no --run-id, E4b uses e4b-block or e4b-pilot, then <base>-2, <base>-3, and so on. The chosen id is the first line of the run and a field on the receipt. A name already used by a receipt, a series directory, a log, a compile cache, or a quarantined run is refused.

Threat model

interlock touches local devices and local HTTP, and the network only to download a model that an experiment spec already named, from that model's publisher, or to check C2 sources when asked.

• Receipts and the routing table are local files.
• Ollama calls, when a run is actually started, go to 127.0.0.1:11434 (the 5090) or 127.0.0.1:11490 (CPU). Cloud model tags are refused.
• OpenVINO runs, when a run is actually started, use NPU, CPU, or the Intel iGPU (vendor 0x8086 and INTEGRATED). They never select GPU.1.
• A 5090 step without --grant does not start. An NPU or iGPU step without --npu-session does not start.
• The 5090 path checks that the VRAM watchdog heartbeat is at most 60 seconds old and that no model is loaded. The heartbeat file is not opened. Only its modified time is read.
• Nothing here listens on a port.
• interlock does not send telemetry.
• The Linux CUDA path (platform: linux-cuda, for remote colab:<card> windows) runs on the remote VM, not on the rig. It starts its own Ollama on a dedicated loopback port (127.0.0.1:11441 by default), refuses the shared ports 11434 and 11490, and refuses any Ollama URL that isn't loopback: the host must be 127.0.0.1:<port>, [::1]:<port>, or localhost:<port> when localhost resolves only to loopback, with the port in ASCII digits from 1 to 65535. It reads the card through NVML or nvidia-smi. It does not download models, and it does not touch Windows counters.
• interlock c2-sources --verify-live is the one command that reaches the internet for C2. It sends read-only HTTPS requests to registry.ollama.ai (manifests and config blobs) and huggingface.co (file trees and tokenizer.json), and nothing else. It downloads no model weights and sends no credentials. Without --verify-live, c2-sources reads only local files.
• A remote window's results reach the table only as a receipt file that interlock import --remote-window reads. interlock does not connect to Colab. A field named for an account (account, user, owner, their _id forms, ggufRepo, gguf_repo or gguf_source) must hold sha256:<64-hex>. Remote rows also refuse a tagged account token, such as account:<id> or user=<id>, in any string value or key, and remote_device refuses account ids, emails, paths and repo handles in its own fields. Ordinary words such as "user-facing" pass in a value or a key.

Layout

devices is the allow-list. guards is the grant, the session, the heartbeat and the empty card. receipt is the schema and interlock check. importers rebuilds table/routing.jsonl from pinned receipts, and importers/remote.py reads a remote window's receipt. interlock table prints that table, filtered by task or device. table/gaps.md is the experiment backlog. The harness runs a committed spec; harness/linux_cuda.py is the Linux CUDA path for remote cards. interlock route reads that table and does not touch a device.

The build brief and the four feedback notes it names are snapshotted in docs/handoff/. The copy in the studio memory store stays authoritative.

Route

interlock route answers from table/routing.jsonl. It does not start a device, read a ledger, or call offrig.

interlock route --task generate --model Phi --thinking off --ctx 1024 --busy 5090

--model matches a substring of the repo or tag. Hyphens and colons are ignored, so Qwen3-8B matches qwen3:8b. A short substring can match an unintended repo. The matched model is printed. --thinking off matches only off. --thinking on matches any recorded thinking level, and the decision keeps that level, so an OpenVINO low arm stays low. low, medium and high match exactly when the request names one. --min-tok-s is the floor on the minimum repeat-slot decode rate. A refusal names that floor min_tok_s. --budget-left-usd is an input, like --busy. A remote row whose worst case exceeds it is a runner-up. With no flag, a remote suggestion is marked budget unchecked. --allow-remote allows remote colab:<card> rows into the candidate pool (default off), and --device filters by an exact device (colab:L4, 5090) or provider wildcard (colab:*). --as-of defaults to today. The policy itself does not read the clock.

Generate ranks by the lowest repeat-slot decode tok/s. Every other task ranks by the highest repeat-slot latency, or by the maximum claim-wall when the row has no repeat slots. The margin is the gap between those conservative bounds, never a median. Ties leave the 5090 free. A row older than 30 days is stale. A row marked provisional, or with fewer than two repeats, is provisional. Both stay eligible and are flagged.

native_binary_rejected, compile_rejected, compile_timeout, hang, device_lost, oom, load_refused, timeout, ctx_exceeded, the unmeasured kinds (sampler_error, generation_aborted, unmeasured) and a row with parity_valid false (parity_invalid) are listed under excluded, with those codes. They are not candidates. Remote rows are filtered out before the outcome loop unless the request opts in (--allow-remote or --device colab:...), so they never appear in a rig-only request's excluded list or refusal reason. A capability outcome is the code shown when the row is also parity-invalid. When no row meets the constraints the command exits 1 and names the constraint that emptied the set. It does not fall back. --json prints a document valid against src/interlock/schema/route-decision.v1.json (marking remote winners with remote: true and remote_device). --max-ttft-ms refuses unless every compared row records the same settings.prefix_caching (on, off, or default). Unset means unknown.

A remote suggestion includes an offrig_plan object when backed by a RunPod offer, or remote: true and remote_device when backed by a colab:<card> row. interlock does not call offrig_plan or offrig_launch, and it does not spend. See docs/offrig-integration.md.

Run an experiment

Every experiment is pre-registered: its spec under experiments/<id>/spec.yaml is committed, with the question, the decision it feeds, the arms, the dataset, the metrics and the stop rules, before any device time. interlock run refuses a spec that was edited after its commit. The spec's blob is checked against that commit before the run (require_unmodified) and printed by --dry-run. It is not written into the receipt. An E4b receipt records the harness git_sha and clean_tree. A fill-card receipt records experiment_spec and run_id, and when preflight ran it also records git_sha and clean_tree. Any other shared-runner receipt records experiment_spec and run_id only. By practice, a rule is not changed after results are seen; a dated amendment comes before the run whose results count, and a run set aside as void is recorded as such.

interlock run experiments/e4/spec.yaml --dry-run
interlock run experiments/e4/spec.yaml --grant "<grant text>" --npu-session "<session>"

A platform: linux-cuda spec (the C2 specs under experiments/c2-*/) runs on a remote Colab VM inside a booked window, against its own Ollama on a dedicated loopback port.

--dry-run prints the plan and the guards a real run would need, and starts nothing. A real run needs the flags its devices call for: --grant for the 5090, --npu-session for the NPU or iGPU, and --cpu-window for a CPU-heavy leg. It writes one receipt under receipts/. interlock import --remote-window <path> then adds a remote Colab window's receipt to table/, and interlock check validates the receipts. A full rebuild from the pinned source repositories needs --source (see below).

Paths and the watchdog

No path of anyone's rig is written in the code. Each location below is an environment variable first, then [paths] in interlock.toml in the working directory, and a value is used as you gave it:

• INTERLOCK_SCRATCH_ROOT / scratch_root: the scratch directory offrig opens as its project for offrig verify calibrate. Default: interlock-scratch under the system temp directory.
• INTERLOCK_WEIGHTS_HOT / weights_hot: the hot weights cache that a spec arm with hf_cache: hot reads its weights from; a missing copy is a refusal. Default: hot/ under HF_HOME, or under the user's Hugging Face cache.
• WATCHDOG_HEARTBEAT / heartbeat: the heartbeat file of the VRAM watchdog, which the heartbeat guard reads before each step on the 5090. There is no default: unset, the run refuses and names WATCHDOG_HEARTBEAT. If you run no watchdog, pass --no-watchdog; the receipt's notes records that.

[paths]
scratch_root = "scratch"
weights_hot = "hf-hot"
heartbeat = "watchdog/HEARTBEAT"

Importing receipts

Most users only merge. interlock import --remote-window <file> adds the rows of a Colab window's receipt (the routing-row JSONL that interlock run writes, or the older {run_id, date, remote_device, records} JSON) to the existing table/routing.jsonl. It needs no other repository, and it refuses a row that is already in the table.

A full rebuild (interlock import with no --remote-window, or with --offers) regenerates the pinned rows from three source repositories, each at a pinned commit: rnd (08ca419f647a3ed69ee645610eaa1c7f35f40f91), npu (275e51101f7557bf50898608812cd61d25d5ff17, on rnd's npu-buildout branch and tagged data/npu-buildout-275e511, so it is a clone of rnd) and aspire (3b05eab928894311993ffa3af2075fef39cccce6). Say where each clone is with --source name=path (repeatable), or once in interlock.toml next to the table:

[import.sources]
rnd = "../rnd"
npu = "../rnd"
aspire = "../aspire-si"

The flag takes precedence over the file, and there is no default path. The commit is not configurable: a clone that does not hold it is refused. The import writes each source's name, pin and path (as you gave it) at the end of table/gaps.md, never into a row. A rebuild replaces only the rows it produces; all the other rows in the table stay, and a rebuild that would drop a row it owns is refused unless you pass --allow-shrink.

When Ollama hangs

interlock ollama-health asks GET /api/version on 127.0.0.1:11434 (the 5090 server) and 127.0.0.1:11490 (the CPU-only server). A port is healthy when that answer arrives within 5 seconds. The command prints one line per port and exits 1 if either is down.

A real interlock run restarts Ollama only on 127.0.0.1:11434, only between steps, and only after a step failed with a transport error, a timeout that is a real hang, or a 5xx while that server is down. Down means three failed health checks, two seconds apart. A request that passed its own deadline is a hang only when health then fails, or /api/ps does not show the model making progress. A long prompt whose version still answers, and whose model is still listed, is not a hang. A 118k-token prompt is not killed for being slow.

Port 11490 is offrig's CPU indexer. Health still reports it. A down 11490 stops the run. interlock never stops that process, and no grant covers a restart of it.

Immediately before anything is stopped, the restart re-checks the --grant, the watchdog heartbeat, and /api/ps. A listing that names a model from another session is refused. An unreadable /api/ps does not, by itself, block a restart of a server that is already down. During a hang, health can still answer. An unreadable /api/ps then refuses the restart, because another session's model would be dropped without being seen.

The process that is stopped is the one listening on 11434. Nothing is killed by name. The tray app stays up unless it is that listener. ollama serve is replayed from the start line recorded for this run: the argv, every OLLAMA_* and LLAMA_ARG_* variable, and the log file. Stdout and stderr are appended to that log. interlock does not sign in. There is no OLLAMA_CUDA_BACKEND variable. If a backend has to be forced, the variable is OLLAMA_LLM_LIBRARY, and it belongs in the recorded start line.

The record is a JSON file named by INTERLOCK_OLLAMA_SERVE. SWITCHYARD_OLLAMA_SERVE, SWITCHYARD_OLLAMA_LOG and SWITCHYARD_OLLAMA_EXE still work for this release and warn. The fields are argv, env, log_path, library_path, ggml_cuda_sha256, and backend_line. One recorded start on this rig is cmd /c ollama serve with OLLAMA_HOST=127.0.0.1:11434 and OLLAMA_MODELS=E:/AI-Models/Ollama, logging to E:/AI/ollama-cu134/system-11434.log. The loaded library is %LOCALAPPDATA%\Programs\Ollama\lib\ollama\cuda_v13\ggml-cuda.dll. The publisher's sha256 of that file starts 6d3128e946bc9c95. A run records the start line it was given and replays that one. A missing record refuses the restart before anything is stopped.

At the start of a run the record's full OLLAMA_* and LLAMA_ARG_* set is compared with the live process environment. That comparison is exact. When the process environment cannot be read, the server's server config log line is the fallback, and that comparison is exact too: a record that is only part of the logged defaults disables restarts. A mismatch means this run makes no restarts. The reason is written on the run marker as ollama_restart_disabled. Nothing is stopped. OLLAMA_HOST matches with or without an http:// or https:// scheme.

After the server is back, the run reads the library sha256 from library_path. Flash attention, the KV cache type, the load_backend line, and the full OLLAMA_* and LLAMA_ARG_* set are read from the log bytes written after the spawn. The bytes before that offset are the previous server, and a line there is not a match. If the log's size cannot be read, that offset is unknown and the run stops. A missing log is empty, and its offset is zero. A load_backend line that has not been written yet is checked after the first model load, and before that sample is kept. A missing or different line stops the run. The new segment's server config line has to cover every variable the run was measured under. Defaults the process did not set may also be logged. A server that comes back on a different library, or under a different environment, stops the run. The driver CUDA version from nvidia-smi is not the check: a restart cannot change it. Flash attention and the KV cache type that the log does not echo fail the match. They are not copied forward from the record and called a read.

If the server is not healthy within 60 seconds, or the card is not back at its baseline, the run stops and says a person has to start Ollama. At most two restarts a run. A failed restart counts against that cap.

Each restart is written on the run marker and on settings.ollama_restarts: when, which port, why (down or hung), the old and new pids, the time to healthy, and the version. The step that was in flight is inconclusive, never a pass. It is tried again only when the spec sets retry (the default is none). A kept retry sets settings.retried. The first kept sample after a restart sets settings.cold_after_restart and stays out of the warm percentiles. A measurement taken across a restart is not kept.

--dry-run prints this policy and does not contact either port.

Develop

Python 3.15 (.python-version); 3.13 and 3.14 also work. uv sync --frozen creates the project environment from the lockfile. It does not share the NPU environment. OpenVINO 2026.4.1 is the intel extra. Tests do not need it.

uv sync --frozen
uv run interlock check
uv run pytest

Contributing

Bugs go to GitHub Issues. A security report follows SECURITY.md and is not a public issue. Changes come in as pull requests against main. The bar, and the commands to run first, are in CONTRIBUTING.md.

Parts of this repository were Written with Grok (xAI). Grok, Kimi and Claude are credited in CONTRIBUTORS.md.

Built by MCP Tool Shop