News
- [2026/11] 🎙️ Voxt 2.0 redefines open-source voice interaction on Mac: voice input, any-to-any translation, and a local AI assistant, with a 92% shorter wait after long dictation than 1.x. [Download] [Changelog]
- [2026/11] 🔁 OpenRSI × SGLang-Omni: recursive self-improvement (RSI) scaled to production AI infrastructure, with 14 merged PRs across 8 speech and omni models and +7.6% to +326.5% throughput on H100. [Blog]
- [2026/10] 🗺️ A unified runtime for interactive intelligence: native full-duplex models, turn-based multimodal models, and cascaded systems, coordinated through composable graphs and persistent sessions. [Blog]
- [2026/10] 🚀 SGLang-Omni v0.2.0 is on PyPI. Install with
uv pip install --prerelease=allow "sglang-omni==0.2.0". [Installation]
- [2026/10] 🐟 Day-0 support for Fish Audio Drama 3: multi-speaker text-to-speech with voice cloning and natural-language control, streamed on
/v1/audio/speech. [Blog] [Cookbook] [Technical Report]
- [2026/10] 🎙️ Introducing Voxt, an open-source voice input tool for Mac, now integrating SGLang-Omni for faster local speech recognition on Apple Silicon. Combine ASR with LLM-based text refinement and personal system prompts for accurate technical terminology and personalized voice input, with fully local or remote inference options. [Download][Demo]
- [2026/09] 🐧 Day-0 support for AuK and AuK-Flash: text + voice instructions → 24 kHz speech on
/v1/audio/speech, audio + editing instructions → edited speech on /generate. [Cookbook]
- [2026/08] 🎵 Day-0 support for MiniMax Music 3: lyrics + caption → 32 kHz stereo song on
/v1/audio/speech. [Cookbook]
- [2026/08] 🚀 TTS architecture refactor: shared pipeline state, engine construction, reference encoding, capability metadata, and vocoder scheduling. [Roadmap] [Blog]
- [2026/06] 🔥 MOSS-TTS Local Transformer v1.5 on SGLang-Omni with native-streaming 48 kHz speech. [Blog] [Cookbook]
- [2026/06] 🔥 Higgs Audio v3 TTS for real-time, controllable speech. [Blog] [Cookbook]
About
SGLang-Omni is a multi-stage serving runtime for omni, speech, and TTS models. Its design target is multi-stage decoding: generation split across heterogeneous stages with different compute patterns, dependency structures, and resource needs. SGLang-Omni owns the pipeline topology, stage lifecycle, inter-stage transport, model-family integration layer, and OpenAI-compatible serving surface, while composing with SGLang for high-performance autoregressive scheduling and model execution where applicable.
- Multi-stage runtime: SGLang-Omni models generation as coordinated stages: preprocessing, encoders, autoregressive engines, talkers, decoders, vocoders, and aggregators.
- Stage-specialized scheduling: Each stage runs behind a scheduler matched to its workload, from SGLang-backed autoregressive scheduling to lightweight preprocessing and streaming vocoder loops.
- Transport-aware execution: A control plane coordinates requests while the relay data plane moves tensor payloads across shared-memory, NCCL, NIXL, and Mooncake backends.
- API surface: OpenAI-compatible endpoints expose multimodal chat, speech generation, batch speech, streaming speech, uploaded voices, and transcription.
This is the beginning of the README. Read the rest at the original. Read the full README ↗
