audio.cpp

audio.cpp is a high-performance C++ audio inference framework built on top of ggml, designed to make modern local audio models practical, portable, and fast.

Tired of juggling a dozen Conda environments, hundreds of Python packages, and dependency conflicts just to try a few audio models? audio.cpp gives those paths a shared native runtime instead. Runs on Windows, Linux, and macOS, with support for NVIDIA, AMD, Apple Silicon, and CPU-only machines.

Lost in a sea of audio model names, papers, and codebases? Explore 100+ audio models in the Audio Model Architecture Atlas to see how they're built, what they share, and where they differ, whether you're learning or building your own.

Supported backends:

[!TIP]

v0.9.4 highlights

Experimental HIP kernels: Select --backend hip --backend-impl gfx1151_v1 to try alternative AMD kernels. On the tested Strix Halo GPU, Q8 generation time fell by 76.7% for Irodori-TTS Small MF, 30.9% for Qwen3-TTS 1.7B VoiceDesign, and 14.0% for BreezeTTS2 versus default HIP. Despite its name, gfx1151_v1 can also run on RDNA1/2, as well as RDNA3/4; available fast paths vary by architecture. Gains vary by model and hardware; the implementation is opt-in. See the HIP guide. Contributions are welcome: test other AMD GPUs, report model performance and correctness, or help improve the kernels. Thanks to AMD DevLab for providing hardware access!

Metal optimizations: New SSM scan, convolution, and dataflow optimizations reduce RE-USE processing time by 56.5% on M4 Pro. Parakeet’s channel-contiguous subsampling and matrix projections reduce end-to-end processing time by 12.1–13.9%, with unchanged transcripts.

Browser inference with WASM + WebGPU: Run supported models locally in the browser. On RTX 5090 in Chrome, measured throughput reaches 23.3× real time for Supertonic 3, 13.9× for Kokoro, 94.3× for SenseVoice ASR, and 25.0× for warmed LFM3-Audio ASR. See the tested models, settings, and performance table.

Experimental streaming avatar voice chat: Talk to LFM3-Audio through the WebUI while SoulX-FlashHead animates your chosen portrait. Responses stream as text, speech, and synchronized head animation, with voice selection, adjustable speaking speed, and a customizable personality.

audio.cpp is built for real end-to-end execution rather than one-off model demos. Its optimized, reusable building blocks support speech generation and recognition, separation, alignment, and multi-model workflows. Shared components make new models easier to integrate and let improvements benefit multiple model families while keeping inference efficient, maintainable, and portable.

Features

⭐ 143 audio model families and 250+ variants. Run TTS, ASR, music generation, voice cloning, conversion, separation, and more locally through a shared CLI and server API.

⭐ Built-in WebUI. Generate and clone voices in Studio, compare models in Arena, manage models and voice libraries, and assemble narration with reusable speakers and passage editing in the TTS app.

This is the beginning of the README. Read the rest at the original. Read the full README ↗