In production
Onde is already shipping in real apps on the App Store and Google Play. Chat runs fully on-device, so there is no server round trip and no user data leaving the device. For SDK docs, platform notes, and setup details, see https://ondeinference.com/sdk. If you want to test downloads, model selection, or GGUF export before wiring the engine into app code, use Onde CLI.
Siti AI is the flagship open reference app — a private, on-device assistant built on Onde, open source under Apache-2.0. Its source is a complete, readable example of wiring the engine into a shipping Tauri app across macOS, iOS, and Android.
Model formats
Onde can load GGUF models and UQFF models through the same chat engine. UQFF is mistral.rs' native pre-quantized format. Point model_id at the UQFF export — the repository or local directory holding the shards, residual.safetensors, config.json, and the tokenizer — and name the first shard (or a shorthand such as q4k) in files. Use the UQFF repository, not the original unquantized one; the export is self-contained and everything is resolved relative to it.
use onde::inference::{ChatEngine, UqffModelConfig};
let engine = ChatEngine::new();
engine
.load_uqff_model(
UqffModelConfig {
model_id: "mistralrs-community/gemma-4-E4B-it-UQFF".into(),
files: vec!["q4k-0.uqff".into()],
display_name: "Gemma 4 E4B (UQFF Q4K)".into(),
approx_memory: "~2.5 GB (UQFF Q4K)".into(),
chat_template: None,
},
None,
None,
)
.await?;
For sharded UQFFs, passing the first shard is enough; mistral.rs discovers sibling shards with the same prefix.
License
Onde is dual-licensed under MIT and Apache 2.0. You may use it under either license at your option.
Dependency attribution
Model licenses
Models downloaded by Onde have their own licenses independent of this crate. By using Onde, you are also subject to the license of the model you load:
¹ Qwen Community License conditions: no training of competing models, attribution required, no misrepresentation of origin. Organisations with more than 100 million monthly active users must obtain a separate commercial licence from Alibaba Cloud.
² DeepSeek License v1.0 conditions: use-based restrictions apply (see Attachment A of the license). Prohibits military use, generation of disinformation, and certain other uses. Governing law is PRC law.
Onde's own license (MIT OR Apache-2.0) is independent of these model licenses. If you build an application on top of Onde, you are responsible for complying with the license of whichever model your users load.
Copyright
© 2026 Splitfire AB (Onde Inference).
