tract

Sonos' neural-network inference engine.

tract loads ONNX and NNEF models, optimises them, and runs them anywhere — from embedded ARM CPUs to NVIDIA / Apple GPUs, in the browser via WebAssembly, or on a Linux / macOS / Windows workstation. It is used in production at Sonos for wake-word and streaming speech-recognition workloads, and also runs LLM, text-to-image, and classical CV models with a particular focus on the translate-once / ship-tiny-runtime story enabled by its NNEF-based intermediate format (tract-OPL).

Quick start

From examples/onnx-mobilenet-v2:

use tract::prelude::*;
tract::impl_ndarray_interop!();

let model = tract::onnx()?
    .load("mobilenetv2-7.onnx")?
    .into_model()?;

// prepare() optimises and compiles the model for the chosen runtime
let runtime = tract::runtime_for_name("default")?;
let runnable = runtime.prepare(model)?;

let result = runnable.run([input.tract()?])?;

The tract crate (api/rs/src/lib.rs) is the authoritative public API. The internal crates (tract-core, tract-nnef, tract-onnx, ...) are not stable surface and shouldn't be depended on directly.

For Python, see the tract package on PyPI.

Examples

examples/ has runnable demos covering the workloads tract targets today:

Example	What
onnx-mobilenet-v2	Minimal CV starter
tflite-mobilenet-v3	TFLite import path
causal_llm	Transformer text generation
nemo-parakeet-asr / nemo-nemotron-streaming-asr	Speech recognition, including streaming via pulsification
stable-diffusion / stable-diffusion-3 / stable-diffusion-xl	Text-to-image
face_detection_yolov8onnx_example / face_similarity_arcface_onnx	Modern object detection / face recognition
wasm-model-bench	Running tract in the browser

Resources

Technical documentation lives under doc/ (start at doc/intro.md); the doc/cli-recipe.md page collects practical CLI recipes. The Sonos engineering blog has a long-form post on tract internals.

Python

tract is also available as the tract package on PyPI, built on top of the same Rust core:

pip install tract

The API mirrors the Rust pipeline: load a model, set input facts, optimise, then run. Documentation: sonos.github.io/tract. Source lives in api/py/.

Runtimes

Runtime	Name	Crate	Notes
CPU (x86, ARMv6/7/8, ARM SVE)	"default"	tract-linalg	Default. Hand-rolled SIMD micro-kernels.
Apple Metal	"metal"	tract-metal	Apple GPUs.
NVIDIA CUDA	"cuda"	tract-cuda	NVIDIA GPUs.
WebAssembly	"default"	via standard wasm32 targets	Browser / WASI deployment.

All runtimes share the TypedModel IR and the same loaders, so a model optimised on one platform can be moved to another.

Benchmarks

Every night, main is benchmarked on a fleet of devices — Cortex-A7 to Cortex-A55 boards, RISC-V boards, x86 and Apple desktops, NVIDIA and Apple GPUs, and WebAssembly — over wake-word, speech recognition, speech enhancement, vision and LLM models. The bench page tracks every model on every device. A few of the trends:

Model	Target	Speed	Trend
GTCRN speech enhancement	Cortex-A55	12.1× since Aug 2026
MobileNet v2, multi-thread	Cortex-A53	3.2× since Aug 2026
Parakeet TDT 0.6B preprocessor	Intel i9-11900KB	14.7× since Jun 2026
Llama 3.2 3B text generation	Apple M1 Max, Metal	1.8× since Apr 2025

Speed is relative to the first week of each series, dashed line at 1×. doc/bench/sparklines.py regenerates the charts.

Streaming and pulsification

tract has first-class support for pulsified inference: a network that operates on full sequences during training is translated into one that processes a fixed-size pulse along its streaming axis at each step. This lets the same model serve both batch evaluation and low-latency real-time inference (wake-word, streaming ASR, ...).

This is the beginning of the README. Read the rest at the original. Read the full README ↗