Onde SDK

---

In production

Onde is already shipping in real apps on the App Store and Google Play. Chat runs fully on-device, so there is no server round trip and no user data leaving the device. For SDK docs, platform notes, and setup details, see https://ondeinference.com/sdk. If you want to test downloads, model selection, or GGUF export before wiring the engine into app code, use Onde CLI.

Siti AI is the flagship open reference app — a private, on-device assistant built on Onde, open source under Apache-2.0. Its source is a complete, readable example of wiring the engine into a shipping Tauri app across macOS, iOS, and Android.

---

Model formats

Onde can load GGUF models and UQFF models through the same chat engine. UQFF is mistral.rs' native pre-quantized format. Point model_id at the UQFF export — the repository or local directory holding the shards, residual.safetensors, config.json, and the tokenizer — and name the first shard (or a shorthand such as q4k) in files. Use the UQFF repository, not the original unquantized one; the export is self-contained and everything is resolved relative to it.

use onde::inference::{ChatEngine, UqffModelConfig};

let engine = ChatEngine::new();
engine
    .load_uqff_model(
        UqffModelConfig {
            model_id: "mistralrs-community/gemma-4-E4B-it-UQFF".into(),
            files: vec!["q4k-0.uqff".into()],
            display_name: "Gemma 4 E4B (UQFF Q4K)".into(),
            approx_memory: "~2.5 GB (UQFF Q4K)".into(),
            chat_template: None,
        },
        None,
        None,
    )
    .await?;

For sharded UQFFs, passing the first shard is enough; mistral.rs discovers sibling shards with the same prefix.

---

License

Onde is dual-licensed under MIT and Apache 2.0. You may use it under either license at your option.

• MIT License
• Apache License 2.0

Dependency attribution

Dependency	License	Author
mistral.rs	MIT	Eric Buehler
UniFFI	MPL-2.0	Mozilla
tokio	MIT	Tokio contributors

Model licenses

Models downloaded by Onde have their own licenses independent of this crate. By using Onde, you are also subject to the license of the model you load:

Model	Size	License	Commercial use
Qwen 2.5 1.5B Instruct (GGUF Q4_K_M)	~941 MB	Qwen Community License	✅ with conditions¹
Qwen 2.5 3B Instruct (GGUF Q4_K_M)	~1.93 GB	Qwen Community License	✅ with conditions¹
Qwen 2.5 Coder 7B Instruct (GGUF Q4_K_M)	~4.4 GB	Qwen Community License	✅ with conditions¹
Qwen 3 1.7B (GGUF Q4_K_M)	~1.3 GB	Apache 2.0	✅
Qwen 3 4B (GGUF Q4_K_M)	~2.7 GB	Apache 2.0	✅
Qwen 3 8B (GGUF Q4_K_M)	~5 GB	Apache 2.0	✅
Qwen 3 14B (GGUF Q4_K_M)	~8.4 GB	Apache 2.0	✅
DeepSeek Coder 6.7B Instruct (GGUF Q4_K_M)	~3.8 GB	DeepSeek License v1.0	✅ with conditions²

¹ Qwen Community License conditions: no training of competing models, attribution required, no misrepresentation of origin. Organisations with more than 100 million monthly active users must obtain a separate commercial licence from Alibaba Cloud.

² DeepSeek License v1.0 conditions: use-based restrictions apply (see Attachment A of the license). Prohibits military use, generation of disinformation, and certain other uses. Governing law is PRC law.

Onde's own license (MIT OR Apache-2.0) is independent of these model licenses. If you build an application on top of Onde, you are responsible for complying with the license of whichever model your users load.

---

Copyright

© 2026 Splitfire AB (Onde Inference).