CrispASR

One C++ binary, 145 backends — 69 of them TTS engines — plus multilingual text translation, zero Python dependencies.

CrispASR started as a fork of whisper.cpp and extends that base into a unified speech engine called crispasr, backed by full ggml C++ runtimes for major open-weights ASR and TTS architectures. One build, one binary, one consistent CLI — pick the backend at the command line or let CrispASR auto-detect it from your GGUF file. See Text-to-Speech for the TTS side.

$ crispasr -m ggml-base.en.bin          -f samples/jfk.wav                    # OpenAI Whisper
$ crispasr -m parakeet-tdt-0.6b.gguf    -f samples/jfk.wav                    # NVIDIA Parakeet
$ crispasr -m canary-1b-v2.gguf         -f samples/jfk.wav                    # NVIDIA Canary
$ crispasr -m voxtral-mini-3b-2507.gguf -f samples/jfk.wav                    # Mistral Voxtral
$ crispasr --backend qwen3 -m auto      -f samples/jfk.wav                    # -m auto downloads
$ crispasr --backend kokoro -m auto --tts "Hello world" --tts-output out.wav  # TTS

No Python. No PyTorch. No separate per-model binary. No pip install. Just one C++ binary and a GGUF file.

Browser: All backends compile to WebAssembly (4.3 MB) via build-wasm.sh. Multithreaded, runs entirely client-side with COOP/COEP headers.

Demo: HuggingFace Space — live transcription + TTS + language detection, auto-deployed from hf-space/.

Ecosystem

Project	What it does
CrispASR	This repo — C++ speech engine. 145 backends (69 TTS), CLI + HTTP server + C-ABI + Python/Rust/Dart/Go/Ruby/Java bindings.
CrisperWeaver	Cross-platform Flutter transcription app built on CrispASR. Desktop + mobile, model browser with download queue, mic capture, SRT/VTT/JSON export, diarization, batch processing. Fully offline.
CrispEmbed	Text-related engine via ggml — same philosophy as CrispASR but for embeddings, retrieval, OCR and OMR, Math and Music Notation. Numerous architectures (XLM-R, Qwen3-Embed, Gemma3, ModernBERT, ...), dense + sparse + ColBERT + reranking. PP-OCR, Tesseract, EasyOCR, InternVL2, etc. Python/Rust/Dart bindings.
Susurrus	Python ASR GUI with 9 backends (faster-whisper, mlx-whisper, voxtral, insanely-fast-whisper, ...). The Python counterpart to CrispASR's C++ approach.

---

Table of contents

• Start here — new to CrispASR? Two commands to your first working audio, no repo clone needed
• Supported backends — ASR + TTS + translation + post-processing + music & audio analysis
• Feature matrix
• Install & build — quick install (full guide in docs/install.md); which prebuilt Linux tarball to download — the -hip / -vulkan builds require the matching GPU driver and do not fall back to CPU (the -cuda tarballs do, since v0.8.30)
• Quick start — ASR
• Troubleshooting — it printed the banner and stopped, reading the exit code, bisect, which Windows zip

---

Start here

Everything below this section is a catalogue — 100+ backends, browse it when you need one. If you just want CrispASR working, this is the whole path. No repo clone, no Python, no model hunting.

New to the project? docs/getting-started.md walks the same path step by step, with the expected output and the three usual first-run failures.

1. Get the binary

Download one file from Releases and unzip it:

Platform	Download	Notes
Windows	crispasr-windows-x86_64-cpu.zip	Needs AVX2 (2013+ Intel / 2015+ AMD). Older CPU → …-cpu-legacy.zip
Windows + NVIDIA	crispasr-windows-x86_64-cuda.zip	Self-contained; a CUDA Toolkit install is not required. CUDA 12.6 legacy build: …-cuda126.zip (Pascal/Volta and newer, through Hopper); CUDA-13-native build: …-cuda13.zip (Turing+)
macOS, Apple Silicon	crispasr-macos-arm64.tar.gz	Metal GPU + Accelerate; crispasr-macos.tar.gz is the arm64 compatibility alias
macOS, Intel	crispasr-macos-x86_64.tar.gz	CPU + Accelerate; requires AVX2/FMA/F16C (Haswell or newer). Older Intel Mac → crispasr-macos-x86_64-cpu-legacy.tar.gz
Linux	crispasr-linux-x86_64.tar.gz	…-cuda.tar.gz / …-vulkan.tar.gz for GPU

Prefer to build it yourself? See Install & build. The -hip and -vulkan builds require the matching driver and do not fall back to CPU; the Linux -cuda tarballs do fall back.

Check it runs — this should print a version banner and exit:

crispasr --version          # Windows: .\crispasr.exe --version

CUDA builds also print cuda toolkit and cuda runtime ABI, so this command distinguishes the CUDA 12 and CUDA 13 packages without inspecting DLLs.

Windows CUDA packages bundle the runtime they were built against. The CUDA 12.6 legacy CLI and library assets use the -cuda126 suffix; their split packages require cuda126-runtime.zip from the same release. Use the included SHA-256 manifest when reusing runtime DLLs. Do not mix CUDA 12.8 binaries with CUDA 12.6 DLLs. The standard -cuda build retains CUDA 12.8 and native Blackwell support. CUDA 12.8 also supports Pascal and Volta in CrispASR; PyTorch's wheel architecture selection is a separate policy (NVIDIA architecture support).

2. Make it speak

-m auto downloads the model on first use (~135 MB here) and reuses it afterwards — nothing to find or install. It lands in ~/.cache/crispasr/ (%USERPROFILE%\.cache\crispasr on Windows).

crispasr --backend kokoro -m auto --tts "The quick brown fox jumps over the lazy dog." --tts-output hello.wav
# crispasr: TTS output written to 'hello.wav' (78000 samples @ 24000 Hz, 3.25 sec)

Play hello.wav. That is the TTS half working.

3. Transcribe it back

crispasr --backend parakeet -m auto -f hello.wav -l en
# crispasr: transcribed 3.2s audio in 0.32s (10.1x realtime)
# The quick brown fox jumps over the lazy dog.

(~467 MB on first run. -l en skips language auto-detection, which would otherwise fetch a small extra model.) Both halves now work — swap in your own .wav and you are running.

Where to go next

You want to…	Go to
Clone a voice from a recording	docs/tts.md — and read the consent rules first; cloning requires --i-have-rights
Pick a better ASR model	Which backend should I pick?
Transcribe Japanese with Haas-ja	Haas-ja usage and models
Transcribe with word timestamps, SRT/VTT	docs/cli.md
Live mic / streaming	docs/streaming.md
Run it as an HTTP server	docs/server.md
See every backend this binary has	crispasr --list-backends

If nothing happened

Add -v to any command for verbose progress, and --dry-run-resolve to print which model files it would open (and whether they're on disk) without loading anything.

If a command printed its banner and then simply stopped — no error, no output file — that is a crash, not a refusal, and the exit code identifies it in one step. See docs/troubleshooting.md.

---

Supported backends

CrispASR ships 145 backends (see the generated feature matrix for the authoritative, auto-counted list) — the majority for transcription/translation, and 69 TTS engines for synthesis. It also ships audio-to-audio S2S backends, including Sidon restoration and the VoxCPM2 AudioVAE speech upscaler; see the feature matrix for the complete capability list. Pick at the CLI with --backend NAME, or omit it to let the binary auto-detect from the GGUF metadata. Jump to the TTS table for the synthesis side.

ASR backends

Backend	Model	Architecture	Languages	License
whisper	ggml-base.en.bin and all OpenAI Whisper variants	Encoder-decoder transformer	99	MIT
whisper	distil-whisper/distil-large-v3	Distilled Whisper: 32L encoder + 2L decoder (6.3x faster)	English	MIT
dolphin	DataoceanAI1/dolphin-cn-dialect-small-streaming (-m dolphin)	E-Branchformer + Transformer decoder + CTC; CTC prefix beam + attention rescoring	Mandarin + Chinese dialects (lang/region predicted)	Apache-2.0
xasr	GilgameshWind/X-ASR-zh-en (-m xasr)	Streaming Zipformer2 transducer (icefall); 160 / 480 / 960 / 1920 ms chunks, realtime WebSocket session	zh, en (punct + case)	Apache-2.0
parakeet	nvidia/parakeet-tdt-0.6b-v3	FastConformer + TDT	25 EU (auto-detect)	CC-BY-4.0
parakeet	oruk/orukeet (-m orukeet)	parakeet-tdt-0.6b-v3 fine-tune, half the encoder's depthwise kernels replaced by fitted Gabor functions	25 EU (auto-detect)	CC-BY-SA-4.0
parakeet	()

Speech-to-speech audio upscaling and restoration

Backend	Model	Architecture	Input / output	License
sidon	KevinAHM/Sidon-GGUF (base sarulab-speech/sidon-v0.1)	w2v-BERT 2.0 predictor + continuous DAC decoder (more)	16 kHz mono → restored 48 kHz mono	MIT
voxcpm2-vae	AudioVAE V2 from openbmb/VoxCPM2, converted with --vae-only	Isolated causal AudioVAE encoder + decoder (more)	16 kHz mono → upscaled 48 kHz mono	Apache-2.0

huggingface-cli download KevinAHM/Sidon-GGUF sidon-v0.1-f16.gguf --local-dir models
crispasr -m models/sidon-v0.1-f16.gguf -f input.wav --s2s --s2s-output restored.wav

python models/convert-voxcpm2-to-gguf.py --input openbmb/VoxCPM2 \
  --output models/voxcpm2-vae-f32.gguf --vae-only
crispasr -m models/voxcpm2-vae-f32.gguf -f input.wav --s2s \
  --s2s-output upscaled.wav

Text-to-Speech models

Synthesis backends, driven by the --tts flag and a --tts-output PATH.wav. See the dedicated Text-to-Speech section below for quick-start commands and engine selection guidance.

Backend	Models	Architecture	Languages	License
bt2-tts	breeze-tts-2	T5Gemma2 text encoder + Qwen3 backbone + 12L depth decoder over 16 codebooks @ 12.5 Hz; voice cloning, plus voice design / direction via --tts-instruct (strengthen with --tts-cfg-scale 4). Codec companion is the shipped qwen3-tts tokenizer. NON-COMMERCIAL weights — needs --accept-license other. Derived from Breeze TTS 2 by BreezeBlue and licensed for research and non-commercial use only.	en, zh	other (BreezeBlue Research & Non-Commercial)
miotts	MioTTS-0.6B	Qwen3 LLM + MioCodec-v2 FSQ codec (25 Hz, 44.1 kHz output)	ja, en	Apache-2.0
arktts	Edge0/Audio8-TTS-Realistic-0.6b	24L AR backbone + 4L RAS fast code predictor + Snake 44.1 kHz vocoder	multilingual	Apache-2.0
vibevoice-tts	VibeVoice-Realtime-0.5B, VibeVoice-1.5B	DPM-Solver++ + σ-VAE decoder; voice presets or cloning	en, zh	MIT
kugelaudio	kugelaudio-0-open	Qwen2.5-7B LM + 4L DiT diffusion + acoustic VAE decoder; voice cloning	multilingual	Apache-2.0
qwen3-tts	, ,

Backend	Voice cloning	Sampling	kHz	Auto-download	Flash attn
vibevoice-tts	yes	temp	24	yes	yes
qwen3-tts	yes*	temp	24	yes	yes
omnivoice	yes	temp	24	—	—
kokoro	—	—	24	yes	—
orpheus	—	temp	24	yes	yes
chatterbox	yes	temp	24	yes	yes
outetts	yes (JSON)	temp	24	yes	yes
indextts	yes	temp	24	yes	yes
voxcpm2-tts	yes	—	48	yes	—
cosyvoice3-tts	yes	temp	24	yes	yes
f5-tts	yes	—	24	yes	—
irodori-tts

* CustomVoice variant only; Base uses baked speakers via --voice <name>.

Output language. -tl <lang> (or -l) selects the language to speak; cosyvoice3-tts, qwen3-tts and moss-tts act on it natively. For cross-lingual cloning — an English reference clip speaking German, the subtitle-dubbing case — also pass -sl <lang> for the language the reference is spoken in, so cosyvoice3 drops the reference transcript instead of carrying its accent. Over HTTP: "language" + "source_lang" on POST /v1/audio/speech. See docs/tts.md.

Translation

Text-to-text translation, distinct from the audio-side --translate flag (which routes audio → English text on whisper / canary / etc.). Driven by --text "..." -sl <src> -tl <tgt>.

Backend	Models	Architecture	Languages	License
m2m100	facebook/m2m100_418M	12L enc + 12L dec transformer, SentencePiece 128K (more)	100 langs, any-to-any	MIT
m2m100-wmt21	facebook/wmt21-dense-24-wide-en-x + facebook/wmt21-dense-24-wide-x-en	Same as m2m100, scaled to 4.7B (24L enc) (more)	English ↔ 7 langs (separate en-x / x-en checkpoints)	MIT
marian	Helsinki-NLP/opus-mt-de-en, opus-mt-en-de — hosted as cstr/opus-mt-<src>-<tgt>-GGUF (f16 + q8_0) for 24 directions: de↔en, fr↔de, fr↔en, ar↔de, ar↔en, it↔de, it↔en, es↔de, es↔en, he↔de, en→he, he→en, tr→en, en→tr (he→en and en→tr are the larger tc-big models); other pairs via models/convert-marian-to-gguf.py	MarianMT 6L enc + 6L dec transformer, d=512, ~75M parameters, SentencePiece unigram + 58K joint vocab (more)	one direction per checkpoint (de→en, en→de)	CC-BY-4.0 (OPUS-MT; attribution required)
madlad

# m2m100 base (production-ready)
./build/bin/crispasr --backend m2m100 -m auto \
    --text "Hello world, how are you today?" \
    -sl en -tl de
# → Hallo Welt, wie bist du heute?

# WMT21 dense (English ↔ X, 4.7B — auto-downloads ~2.5 GB).
# Two separate checkpoints: en-x for English-source, x-en for
# English-target. Pick the one matching your `-sl`/`-tl` direction
# (or pass an explicit `-m <path>` to load the other manually).
./build/bin/crispasr --backend m2m100-wmt21 -m auto \
    --text "The president said he would not attend." \
    -sl en -tl de   # uses wmt21-dense-24-wide-en-x

./build/bin/crispasr --backend m2m100-wmt21 \
    -m models/wmt21-dense-24-wide-x-en-q4_k.gguf \
    --text "Le président a dit qu'il ne serait pas présent." \
    -sl fr -tl en   # uses wmt21-dense-24-wide-x-en

# Opus-MT / MarianMT (one small model per direction: `-m auto` is de→en,
# `-m opus-mt-en-de` the other way; ~84 MB, CC-BY-4.0)
./build/bin/crispasr --backend marian -m auto \
    --text "Die Konferenz findet am 3. Oktober in Berlin statt." \
    -sl de -tl en --beam-size 1
# → The conference will take place on 3 October in Berlin.

# MADLAD-400 3B (419 languages, bit-token-identical to Python SP)
./build/bin/crispasr --backend madlad -m auto \
    --text "Hello world." \
    -sl en -tl ta

For 2-stage pipelines (e.g., ASR → m2m100), use the dedicated --tr-sl / --tr-tl flags; they fall back to -sl / -tl when unset, so single-stage standalone usage is just -sl/-tl.

Post-processing models

Work with all backends.

Model	Task	Architecture	Languages	License	HuggingFace
FireRedPunc	Punctuation restoration	BERT-base (12L, d=768), 5 classes	Chinese + English	Apache-2.0	cstr/fireredpunc-GGUF
fullstop-punc	Punctuation restoration	XLM-RoBERTa-large (24L, d=1024), 6 classes	EN, DE, FR, IT	MIT	cstr/fullstop-punc-multilang-GGUF
punctuate-all	Punctuation restoration	XLM-RoBERTa-base (12L, d=768), 6 classes	12 languages	MIT	cstr/punctuate-all-GGUF
PCS	Punc + truecase + SBD	XLM-RoBERTa-base (12L), 4 heads	47 languages	Apache-2.0	--punc-model pcs
truecaser‑lstm	German truecasing (best)	BiLSTM char-level (2×150, 3.2 MB, 97.9% F1)	German	Apache-2.0	--truecase-model lstm
truecaser‑crf	German truecasing	CRF + context features (8.5 MB)	German	MIT	--truecase-model crf
truecaser‑de	German truecasing (simple)	Statistical word-frequency (71K entries, 1.7 MB)	German	MIT	--truecase-model auto
CLD3	Text language ID	Embedding-bag → FC + ReLU → softmax (~1.5 MB F32)	109 ISO 639-1	Apache-2.0	cstr/cld3-GGUF

Audio codecs

Shared codec modules used by TTS backends. Also available standalone for encode/decode.

Model	Architecture	Sample Rate	Token Rate	License	HuggingFace
MioCodec v2	WavLM encoder → FSQ(12800) → Transformer decoder + AdaLN-Zero + SnakeBeta upsampler + iSTFT	44.1 kHz	25 Hz (341 bps)	MIT	cstr/miocodec-v2-44k-GGUF
SNAC 24 kHz	3-codebook RVQ + decoder blocks (stride 8/8/4/2)	24 kHz	3×12.5 Hz	MIT	cstr/snac-24khz-GGUF

All runtimes share ggml-based inference. The speech-LLM backends (qwen3, voxtral, voxtral4b, granite, glm-asr, kyutai-stt) inject audio encoder frames directly into an autoregressive language model's input embeddings, instead of using a dedicated CTC/transducer/seq2seq decoder. The fastconformer-ctc backend hosts the NeMo FastConformer-CTC standalone ASR family — stt_en_fastconformer_ctc_{large,xlarge,xxlarge} and the architecturally-identical parakeet-ctc-{0.6b,1.1b} (different training data + tokenizer, same encoder + head shape) — with greedy CTC decoding. Same C++ runtime as the canary-ctc aligner.

Music & audio analysis

Beyond speech, CrispASR runs several music/audio analysis tasks — each a small GGUF with the architecture auto-detected, no Python. See docs/cli.md for the per-task flags and output formats.

• Source separation (--separate) — split a mix into stems (<input>_<stem>.wav) via mel-band-roformer (vocal/instrumental, MIT), htdemucs (4-stem), or mini-bs-roformer (HiDolen V2.5, 46.2M parameters, stereo bass/drums/other/vocals at 44.1 kHz). --stems vocals,drums selects a subset; --sep-output-dir sets the output location. Mini BS RoFormer uses an explicitly converted local GGUF. The SCNet / SCNet-large candidate backend adds native four-stem separation with local F16/Q8_0/Q4_K GGUFs; original-checkpoint fidelity and paired performance remain qualification gates.
• Piano transcription (--backend piano-transcription) — piano audio → note events (88 keys @ 100 fps, ByteDance/Kong CRNN; F16 GGUF ≈ 77 MB).
• Polyphonic note events (--backend basic-pitch) — Spotify Basic Pitch, any instrument → note events (~110 KB model).
• Multi-instrument transcription (--backend mt3, alias music-transcription) — MT3's T5 encoder/decoder emits note events with per-instrument programs (F16 GGUF ≈ 96 MB).
• Onsets & Frames (--backend onsets-and-frames) — Hawthorne et al.'s piano transcriber (MIT), the best solo-piano model here per megabyte: 69.0% note F1 on MusicNet's piano pieces against Basic Pitch's 57.5%, and F1-identical to its ONNX export at f32 and Q8_0. Q4_0 GGUF 18.6 MiB, Q8_0 30.8 MiB, F32 101.9 MiB. See docs/music-transcription/ONSETS_AND_FRAMES.md.
• hFT-Transformer (--backend hft-transformer) — Toyama et al.'s hierarchical frequency-time transformer (MIT), the most accurate solo-piano model here and by far the smallest: 70.5% note F1 on MusicNet's piano pieces from 5.5 M parameters, Q8_0 GGUF 7.0 MiB, Q4_0 4.5 MiB, F32 21.8 MiB. It is also much the most expensive to run — 249 GFLOP of matrix multiply per 2 s of audio, and quantisation makes it slower, not faster, on a CPU without int8 dot-product instructions. Read docs/music-transcription/HFT_TRANSFORMER.md before choosing it over .

Feature matrix

Run crispasr --list-backends to see it live. Each backend declares capabilities at runtime; if you ask for a feature the selected backend does not support, CrispASR prints a warning and silently ignores the flag.

Sortable / filterable view: docs/feature-matrix.html — click any column header to sort, type to filter rows, click cap pills to require a capability. Generated from crispasr --list-backends-json (single source of truth — drift impossible). Regenerate via python tools/gen-feature-matrix.py. A Markdown twin lives at docs/feature-matrix.md.

The static table below is a curated subset focusing on the ASR backends and the cross-cutting features that matter for ASR pipelines. The full 119-backend × 27-cap surface is in the generated views.

Feature	whisper	parakeet	canary	cohere	granite	granite‑4.1	voxtral	voxtral4b	qwen3	fc‑ctc	wav2vec2	glm‑asr	kyutai‑stt	firered	moonshine	moon‑stream	omniasr	omniasr‑llm	vibevoice	gemma4‑e2b	mimo‑asr	funasr	paraformer	sensevoice
Native timestamps	✔	✔	✔	✔									✔
CTC timestamps			✔		✔	✔	✔	✔	✔	✔	✔	✔		✔	✔	✔	✔	✔	✔	✔	✔	✔	✔	✔
Word-level timing

The matrix above covers 24 ASR backends. Additional ASR backends not shown: nemotron (39-lang streaming ASR with cache-aware FastConformer + RNN-T), lfm2-audio (ASR + TTS + S2S in one model), moss-audio (audio understanding + ASR), moss-transcribe (Qwen3-Omni encoder + Qwen3-1.7B ASR), hojo-asr (Qwen3-Omni encoder + Conformer adapter + Qwen3-4B multilingual ASR), mini-omni2 (ASR + TTS + S2S), kugelaudio (7B audio understanding). See docs/feature-matrix.md for the full 109-backend matrix. TTS-only backends (kokoro, qwen3-tts + variants, vibevoice-tts, orpheus + DE variants, chatterbox / chatterbox-turbo / chatterbox-nano / kartoffelbox-turbo / lahgtna-chatterbox, dia, bark, outetts, zonos, csm, f5-tts, irodori-tts, supertonic, parler-tts, speecht5, piper, fastpitch, pocket-tts, melotts, cosyvoice3, voxcpm2, tada-tts) all carry the TTS, AUTO_DOWNLOAD, TEMPERATURE, and FLASH_ATTN caps; per-backend cloning + voice-pack support is documented in the Text-to-Speech models table above and docs/tts.md. The vibevoice and lfm2-audio columns mark dual-mode (ASR + TTS) backends.

Nemotron managed downloads use pinned F16 (~1.3 GB); the former Q4 default fails the fixed CPU/T4 original-reference controls. See the default qualification and existing-release usage.

Key: ✔ = native/built-in, -am = via CTC forced aligner (-am canary-ctc-aligner.gguf or -am qwen3-forced-aligner.gguf), LID = via external language identification pre-step (-l auto), pp = via --punc-model post-processor (FireRedPunc or fullstop-punc), * = experimental or partial support, † = PLUS variant only (native [T:N] word timestamps with -owts; base uses -am). granite-4.1 covers both the regular and -plus variants; granite-4.1-nar is a non-autoregressive variant with encoder+projector only (no LLM decode features). The KV quant row marks backends that honor CRISPASR_KV_QUANT={f16,q8_0,q4_0} — CTC-style backends without a KV cache (parakeet, fc-ctc, wav2vec2, kyutai-stt, firered, moonshine variants, omniasr-CTC) don't apply. The same backends also honor the per-half CRISPASR_KV_QUANT_K / CRISPASR_KV_QUANT_V overrides (llama.cpp --cache-type-k / --cache-type-v parity) for asymmetric K-vs-V precision; common recipe K=q8_0 V=q4_0 saves ~40 % more KV memory than symmetric Q8_0. The mmap weights row marks backends consuming core_gguf::load_weights() and therefore honoring CRISPASR_GGUF_MMAP=1; whisper itself uses upstream's loader and is unaffected. See docs/cli.md Memory footprint for usage + recommended combos.

Speaker diarization as a post-processing step via --diarize:

• energy / xcorr — stereo-only, no extra deps
• foxnose — best accuracy, no external deps: WeSpeaker ResNet34-LM embeddings + GMM/BIC speaker counting + spectral clustering + Viterbi smoothing. Estimates the speaker count rather than needing it up front; --diarize-embedder auto fetches the GGUF (24 MB, CC-BY-4.0). 7.3 % DER on VoxConverse dev where pyannote + TitaNet scores 7.8 %, and 3.18 % vs the upstream reference's 3.07 % when scored on turns (more)
• sortformer — NVIDIA Nemotron-3-Diarization (streaming Sortformer v3, #466): one end-to-end network, up to 8 speakers in order of appearance, overlap-aware, no embedder or clustering. --diarize-model auto fetches NVIDIA's 107 MB q8_0 GGUF (OpenMDW-1.1, commercial use allowed). Matches transformers exactly on every p > 0.5 decision (more)
• pyannote — native GGUF (no Python, no sherpa-onnx); add --diarize-embedder auto (TitaNet) or --diarize-embedder indextts (ECAPA-TDNN) for globally stable speaker IDs across long files
• sherpa / ecapa — external sherpa-onnx subprocess; runs once globally on full audio for consistent speaker IDs (#110)
• vad-turns — mono-friendly gap-based proxy

The server endpoint supports response_format=diarized_json for structured speaker-labelled output with normalised speaker letters (A, B, C …) — see docs/server.md.

Full reference + tuning knobs (cluster threshold, max speakers, pluggable embedder adapters): see docs/cli.md#diarization.

Language identification for backends without native LID: --lid-backend whisper (default, 75 MB ggml-tiny.bin), --lid-backend silero (native GGUF, 16 MB, 95 languages), or --lid-backend firered (FireRedLID, 1.7 GB, 120 languages — Conformer encoder + Transformer decoder).

Voice activity detection: --vad uses the default Silero VAD (~885 KB, auto-downloaded). Each VAD segment is transcribed independently, producing separate SRT/VTT entries with correct timestamps. Use --vad --split-on-punct for best subtitle output. Four VAD backends: Silero (default), FireRedVAD (-vm firered, recommended), MarbleNet (-vm marblenet, 439 KB, 6 languages), Whisper-VAD-EncDec (-vm whisper-vad, experimental).

Punctuation restoration (--punc-model): CTC-based backends output lowercase without punctuation. Named shortcuts: auto/firered (Chinese+English), fullstop (EN/DE/FR/IT, XLM-R-large), punctuate-all (12 languages, XLM-R-base), pcs (47 languages, punc + truecasing + sentence boundary detection in one model). Or pass a GGUF path directly. Also available via Python/Rust/Dart wrappers (crispasr.PuncModel).

Truecasing (--truecase-model): Restore German noun/name capitalization in lowercase ASR output. Three options in ascending quality: auto (statistical, 1.7 MB), crf (CRF with context, 8.5 MB), lstm (BiLSTM char-level, 3.2 MB, recommended — 97.9% F1, handles adjective/noun distinction and formal "Ihnen"). All auto-download from cstr/truecaser-de. Or use --punc-model pcs for neural punc + truecasing in one pass (47 languages).

Backend	Punctuation	Capitalization	Notes
whisper	✔	✔	Full punctuation and casing
parakeet	✔	✔
canary	✔	✔
cohere	✔	✔	Toggleable via --no-punctuation
granite	✔	✔	LLM output
voxtral	✔	✔	LLM output
voxtral4b	✔	✔	LLM output
qwen3	✔	✔	LLM output
funasr	✔	✔	LLM output (Qwen3-0.6B decoder). Chinese chars carry full-width period; mlt-nano variant adds Latin-script casing + punctuation.
sensevoice	✔	✔	CTC output with native ITN — toggle off via --no-punctuation, which controls Arabic-digit vs spelled-out numerals + comma/period emission.
paraformer	no	no	NAR character-level output — add --punc-model
gigaam	✔ (e2e_*)	✔ (e2e_*)	The e2e_* revisions carry punctuation + casing + inverse text normalization in the SentencePiece vocabulary. The charwise ctc / rnnt revisions emit lowercase Cyrillic with no punctuation — but auto-restoration is still suppressed for them, because the auto-enabled FireRedPunc is a Chinese/English model and injects full-width CJK punctuation into Russian. Use an e2e_* revision for punctuated output, or pass an explicit --punc-model.
glm-asr	✔

Other freely-licensed alternatives that could be added: felflare/bert-restore-punctuation (MIT, English, includes truecasing), xashru/punctuation-restoration (Apache-2.0, 40+ languages, BiLSTM-CRF).

Progressive subtitle output (--flush-after): By default, non-whisper backends buffer all segments and print output at the end. For real-time subtitle consumption (PotPlayer, custom media players), use --flush-after 1 to print each SRT entry to stdout immediately after its VAD segment is transcribed:

crispasr --backend parakeet -m parakeet.gguf --vad --flush-after 1 -osrt -f long_audio.wav
# SRT entries appear progressively as each segment finishes

JSON output with language detection: When using -l auto -oj, the JSON output includes detected language info:

{
  "crispasr": {
    "backend": "cohere",
    "language": "en",
    "language_detected": "en",
    "language_confidence": 0.977,
    "language_source": "ecapa"
  },
  "transcription": [...]
}

Which backend should I pick?

Need	Pick
Battle-tested, all features exposed	whisper
Lowest English WER	cohere
Fastest (16x realtime on CPU)	moonshine (tiny), fc-ctc (10x)
Multilingual + word timestamps + fast	parakeet (2.9x RT)
Multilingual with explicit language control	canary
Speech translation (X→en or en→X)	canary, voxtral, qwen3
30 languages + Chinese dialects	qwen3
1600+ languages	omniasr (CTC or LLM)
Realtime streaming ASR (native incremental encoder, ~2× RT feed; sub-second-token target deferred to phase 2)	voxtral4b
Highest-quality offline speech-LLM	voxtral
Apache-licensed speech-LLM	granite, voxtral, qwen3, omniasr-llm
Lightweight CTC-only (fast, no decoder)	wav2vec2, fc-ctc, data2vec, omniasr
Russian	gigaam (e2e_rnnt — 8.4 % avg WER, punctuation + ITN), whisper, qwen3
Mandarin + Chinese dialects	firered-asr, qwen3, glm-asr, funasr, paraformer, sensevoice
Multilingual (31 langs) speech-LLM	fun-asr-mlt-nano, qwen3, omniasr-llm, gemma4-e2b
Multilingual (50+ langs) + LID + audio-event in one pass

CPU performance tips

Audio-LLM backends (qwen3, voxtral, granite, glm-asr, etc.) run full transformer decoder stacks (28+ layers, 2048-dim) and are dramatically slower on CPU than encoder-only backends. On older dual-core hardware they can drop below 0.01× realtime. If you're on CPU-only hardware:

• Prefer moonshine (16× RT), fc-ctc (10× RT), parakeet (2.9× RT), or whisper for usable speeds.
• Use --flush-after 1 to see results as each VAD slice completes instead of waiting for the entire file.
• Use -pp / --print-progress for per-slice progress indicators on all backends (unified backends show slice-level progress; whisper shows encoder-level progress).
• Quantize models to Q4_K or Q5_K to reduce memory and compute.

Language detection for backends that don't do it natively

Cohere, canary, granite, voxtral and voxtral4b need an explicit language code up front. If you don't know the language, pass -l auto and crispasr runs an optional LID pre-step before the main transcribe() call:

# Downloads ggml-tiny.bin (75 MB, 99 languages) on first use
crispasr --backend cohere -m $TC/cohere-transcribe-q5_0.gguf \
         -f unknown.wav -l auto
# crispasr[lid]: detected 'en' (p=0.977) via whisper-tiny
# crispasr: LID -> language = 'en' (whisper, p=0.977)

These LID providers are available:

• --lid-backend whisper (default) — uses a small multilingual ggml-*.bin model via the crispasr C API. Auto-downloads ~75 MB on first use. 99 languages.

• --lid-backend silero — native GGUF port of Silero's 95-language classifier. 16 MB F32. Runs as a ggml graph (multi-threaded SIMD on CPU, GPU offload on Metal/CUDA; on Vulkan the graph is routed to CPU pending an upstream kernel fix). Analyzes the first 30 s of audio (CRISPASR_SILERO_LID_MAX_S overrides); CRISPASR_SILERO_LID_LEGACY=1 restores the old scalar path.

• --lid-backend ecapa — recommended: ECAPA-TDNN (Apache-2.0). Purpose-built for language ID. Very high accuracy on TTS benchmark. Two variants via --lid-model:

  • cstr/ecapa-lid-107-GGUF — VoxLingua107, 43 MB F16, 107 languages, ISO codes (en, de, ...). Default.
  • cstr/ecapa-lid-commonlanguage-GGUF — CommonLanguage, 40 MB F16, 45 languages, full names (English, German, ...).

• --lid-backend firered — FireRedLID (Conformer encoder + Transformer decoder). Q4_K (544 MB), 120 languages including Chinese dialects. Slower but covers more languages.

• --lid-backend probe — no second model at all: ask the ASR model itself. Transcribes a 20 s clip once per language the model declares and keeps the best-scoring candidate (length × text-LID agreement × distinct-token ratio², the last term catching the repetitive output a wrong-language prompt produces). Currently implemented by cohere. The reason to prefer it is correctness, not just the saved download: an external detector knows 99 languages while Cohere Transcribe accepts 14 — and its Arabic finetune only en/ar — so external LID regularly returns a language the model was never trained on, and Cohere answers a wrong language fluently rather than failing. The probe cannot. Cost is one encode + one short decode per candidate, so it runs automatically only for a model with ≤ 4 languages (CRISPASR_COHERE_PROBE_MAX_LANGS); CRISPASR_COHERE_PROBE_TEXTLID=0 drops the text-LID agreement term.

  The ceiling is about cost, not accuracy. Measured on the real models: the two-language Arabic finetune picks ar for an Arabic clip (p=0.675) and for (p=0.647); the 14-language base model, probed across all 14, also gets both right ( p=0.169, p=0.254) — it is simply slower than an external detector. The encoder output is language-independent, so the probe encodes and decodes per candidate (encode is ~87 % of a pass); a 14-candidate probe measured against one-encode-per-candidate, byte-identical output. restores the naive path.

These VAD providers are available:

• Silero VAD (default) — ~885 KB, auto-downloaded via --vad. Industry-standard, well-tested.
• FireRedVAD — DFSMN-based, 2.4 MB, F1=97.57%. Pass --vad -vm firered to auto-download. Recommended.
• MarbleNet — NVIDIA 1D separable CNN, 439 KB, 6 languages (EN/DE/FR/ES/RU/ZH). Pass --vad -vm marblenet to auto-download. Smallest model. (cstr/marblenet-vad-GGUF)
• Whisper-VAD-EncDec (experimental) — Whisper-base encoder + TransformerDecoder head, 22 MB Q4_K. Trained on Japanese ASMR; may not generalise well to all domains. Pass --vad -vm whisper-vad. Slower than others (~1s vs ~50ms). (cstr/whisper-vad-encdec-asmr-GGUF)

Pass --lid-backend off to skip LID entirely.

By default, the external detector checks the first 15 seconds. Pass --lid-full-recording with --detect-language to classify each 15-second window and return one language for the recording. The same opt-in applies to external or self-probe LID during -l auto transcription. The final language is chosen by the duration of non-silent windows; ties keep the first language seen. Near-silent windows are skipped. A final fragment under five seconds is classified with preceding context but counted by its own duration. The reported confidence is a support estimate, not a calibrated recording-wide posterior. It takes longer on long recordings and reports one dominant language, not language changes over time. For standalone detection, no ASR model is needed:

Native providers reuse a loaded detector across windows. The sherpa-ONNX subprocess fallback launches and loads its model once per window, so this mode can be substantially slower with that provider.

crispasr --detect-language --lid-full-recording -f recording.wav

Text language identification (post-ASR / standalone)

Audio LID (above) tags what was spoken; text LID tags what was written. Text LID runs on a transcript or any UTF-8 string and is useful for routing post-ASR pipelines (translation, punctuation, sub selection) without re-running an audio model. Three GGUF families, one binary — the dispatcher picks by general.architecture:

Backend	Labels	Size (F16)	License	HF repo
CLD3 (Google compact language detector v3)	109 ISO 639-1	440 KB	Apache-2.0	cstr/cld3-GGUF
GlotLID-V3 (cis-lmu fastText)	2102 ISO 639-3 + script	250 MB	Apache-2.0	cstr/glotlid-GGUF
LID-176 (Facebook fastText)	176 ISO 639-1	63 MB	CC-BY-NC-4.0¹	cstr/fasttext-lid176-GGUF

¹ LID-176 is CC-BY-NC-4.0 — non-commercial use only. CLD3 + GlotLID-V3 are Apache-2.0 with no such constraint. Pick CLD3 for the smallest, fastest path; GlotLID for maximum coverage (low-resource languages); LID-176 only if you need its specific 176-label space and accept its non-commercial terms.

Standalone CLI — auto-routes by GGUF arch, with auto-download:

crispasr-lid -m auto --text "Bonjour le monde"        # → cstr/cld3-GGUF (default, ~440 KB)
crispasr-lid -m auto:glotlid --text "Bonjour le monde" -k 5
crispasr-lid -m auto:lid-fasttext176 --text "Hallo Welt"
# Or pass an explicit path / canonical filename (looked up in the registry):
crispasr-lid -m cld3-f16.gguf --text "你好世界"
# zh	0.997816
echo "Привет мир" | crispasr-lid -m auto --quiet
# ru	0.907322

Post-ASR pipeline — --lid-on-transcript runs the same dispatcher on the assembled transcript (also accepts auto[:variant]):

crispasr -m ggml-tiny.bin -f speech.wav --lid-on-transcript auto
# (transcript on stdout)
# lang=de	conf=0.997123	backend=lid-cld3

The dispatcher (src/text_lid_dispatch.{h,cpp}) is a thin C ABI façade — one integer compare per call; per-stage diff harness is green at cos≥0.999 across 8 multilingual smoke samples.

---

Install & build

Don't want to build? Prebuilt binaries for Windows, macOS and Linux are on the releases page — see Start here for which file to take. The rest of this section is for building from source.

git clone --recursive https://github.com/CrispStrobe/CrispASR
cd CrispASR
# already cloned without --recursive? initialize the bundled ggml submodule:
#   git submodule update --init --recursive
cmake -B build -DCMAKE_BUILD_TYPE=Release
cmake --build build -j$(nproc)

The ggml/ submodule is required. If you cloned without --recursive, run git submodule update --init --recursive first — otherwise CMake stops with a message telling you to do exactly that.

Produces build/bin/crispasr (main CLI), build/bin/crispasr-quantize, and build/bin/crispasr-diff. No Python, PyTorch, or pip required at runtime — just a C++17 compiler and CMake 3.14+.

For GPU acceleration, add the matching ggml flag at configure time:

cmake -B build -DCMAKE_BUILD_TYPE=Release -DGGML_CUDA=ON     # NVIDIA
cmake -B build -DCMAKE_BUILD_TYPE=Release -DGGML_METAL=ON    # Apple Silicon
cmake -B build -DCMAKE_BUILD_TYPE=Release -DGGML_VULKAN=ON   # cross-vendor

See docs/install.md for the full guide: all GPU backends (CUDA / Metal / Vulkan / MUSA / SYCL), Windows convenience scripts, ffmpeg ingestion, optional BLAS, glibc notes, and the scripts/dev-build.sh wrapper.

If a build runs but the binary exits with no output, see docs/troubleshooting.md.

---

Quick start

Deeper ASR examples below. If this is your first run, use Start here instead. For TTS, the runnable guide is docs/tts.md (Text-to-Speech below is the model catalogue).

Whisper (historical path, byte-identical to upstream whisper.cpp)

# Download a whisper model (same as upstream whisper.cpp)
./models/download-ggml-model.sh base.en

./build/bin/crispasr -m models/ggml-base.en.bin -f samples/jfk.wav
# [00:00:00.000 --> 00:00:07.940]   And so my fellow Americans ask not what your country can do for you
# [00:00:07.940 --> 00:00:10.760]   ask what you can do for your country.

Parakeet (multilingual, free word timestamps, fastest)

# Grab the quantized model (~467 MB)
curl -L -o parakeet.gguf \
    https://huggingface.co/cstr/parakeet-tdt-0.6b-v3-GGUF/resolve/main/parakeet-tdt-0.6b-v3-q4_k.gguf

./build/bin/crispasr -m parakeet.gguf -f samples/jfk.wav
# Auto-detected backend 'parakeet' from GGUF metadata.
# And so, my fellow Americans, ask not what your country can do for you, ask what you can do for your country.

# Word-level timestamps (one line per word)
./build/bin/crispasr -m parakeet.gguf -f samples/jfk.wav -ml 1

Canary (explicit language, speech translation)

# Canary 180M Flash: use the existing handy-computer transcribe.cpp GGUF directly.
# Q5_K_M is the smallest quant locally validated for both ASR and translation.
huggingface-cli download handy-computer/canary-180m-flash-gguf \
    canary-180m-flash-Q5_K_M.gguf --local-dir .

# English ASR with punctuation and capitalization
./build/bin/crispasr --backend canary \
    -m canary-180m-flash-Q5_K_M.gguf -f samples/jfk.wav -sl en -tl en

# English ASR without punctuation/capitalization
./build/bin/crispasr --backend canary \
    -m canary-180m-flash-Q5_K_M.gguf -f samples/jfk.wav \
    -sl en -tl en --no-punctuation

# English speech → German text (use Q5_K_M or higher)
./build/bin/crispasr --backend canary \
    -m canary-180m-flash-Q5_K_M.gguf -f samples/jfk.wav -sl en -tl de

# Legacy Canary 1B v2 remains supported and is the canary registry default.
# Transcription (source == target)
./build/bin/crispasr --backend canary -m canary-1b-v2-q5_0.gguf -f audio.de.wav -sl de -tl de

# Translation (German speech → English text)
./build/bin/crispasr --backend canary -m canary-1b-v2-q5_0.gguf -f audio.de.wav -sl de -tl en

# ...or use the familiar crispasr flag:
./build/bin/crispasr --backend canary -m canary-1b-v2-q5_0.gguf -f audio.de.wav -l de --translate

Canary 180M Flash supports ASR in en, de, es, and fr, plus only the English-pivot translation pairs EN↔DE/ES/FR. It requires an explicit source language and does not provide language detection or native streaming. Its small GGUFs are especially suitable for mobile packaging. Inputs through 40 seconds run directly; longer files use checkpoint-specific offline 20-second windows with 6-second overlap and centered stitching. That path preserves four repeated JFK utterances and monotonic runtime timings in the current regression fixture, but is not native model streaming or a broad long-audio quality benchmark. See the CLI guide for quantization, long-form, and timestamp caveats.

Voxtral (speech-LLM with auto-download)

# First run downloads ~2.5 GB to ~/.cache/crispasr/ via curl, then runs
./build/bin/crispasr --backend voxtral -m auto -f samples/jfk.wav

# Subsequent runs use the cached file
./build/bin/crispasr --backend voxtral -m auto -f samples/jfk.wav -l en

Qwen3-ASR (30 languages + Chinese dialects)

# 0.6B (default, ~500 MB)
./build/bin/crispasr --backend qwen3 -m auto -f audio.zh.wav

# 1.7B (higher quality, ~1.3 GB) — supports both -hf and non-hf source models
./build/bin/crispasr --backend qwen3 -m qwen3-1.7b --auto-download -f audio.wav

# Japanese anime/galgame fine-tune (~1.3 GB)
./build/bin/crispasr --backend qwen3 -m qwen3-ja-anime --auto-download -f anime.wav

Long audio: the default is safe 30 s chunking. --chunk-seconds 0 decodes the whole file in ONE pass (matches the reference model verbatim on multi-minute clips, #218) — but the encoder's full attention is O(N²) in audio length, so keep single-pass clips under ~10 minutes on 16 GB machines. For long-form use prefer the plain -q4_k/-q8_0 GGUFs over the -imatrix variants (see the model card).

GLM-ASR-Nano (Mandarin + dialects + Cantonese + English, 1.5B)

./build/bin/crispasr --backend glm-asr -m auto -f audio.wav

# Long audio in one pass (up to 655 s — 30 s encoder windows, one LLM prompt,
# same layout as the HF/zai reference; matches it verbatim on the #218 clip):
./build/bin/crispasr --backend glm-asr -m auto --chunk-seconds 0 -f long.wav

Note: in single-pass mode the model (like the reference) skips leading non-speech audio; the default 30 s-chunked mode transcribes more of such clips. Custom --ask / non-English --language instructions need a GGUF with baked BPE merges (re-published 2026-07; older GGUFs fall back to the default transcription prompt with a warning).

MiMo-V2.5-ASR (Mandarin + dialects + English, 7.5B Qwen2 LM)

# Download the LM + audio tokenizer (the tokenizer is a separate model)
huggingface-cli download cstr/mimo-asr-GGUF mimo-asr-q4_k.gguf \
    --local-dir ~/.cache/crispasr
huggingface-cli download cstr/mimo-tokenizer-GGUF mimo-tokenizer-q4_k.gguf \
    --local-dir ~/.cache/crispasr

# Transcribe (auto-discovers tokenizer if it sits next to the LM)
./build/bin/crispasr \
    --backend mimo-asr \
    -m ~/.cache/crispasr/mimo-asr-q4_k.gguf \
    --codec-model ~/.cache/crispasr/mimo-tokenizer-q4_k.gguf \
    -f samples/jfk.wav
# Output: And so, my fellow Americans, ask not what your country can do
# for you. Ask what you can do for your country.

The 4.5 GB Q4_K is the recommended quant; F16 (14.9 GB) needs ~16 GB RAM during inference. JFK matches the upstream Python MimoAudio.asr_sft reference verbatim; performance on M1+Metal is ~0.3× realtime (Q4_K dequant per step is the bottleneck — F16 + KV-reuse follow-ups are queued under PLAN #51a/b/c).

Wav2Vec2 (lightweight CTC, any HF Wav2Vec2ForCTC model)

# English (Q4_K quantized, 212 MB — 6x smaller than F16)
curl -L -o wav2vec2-en-q4k.gguf \
    https://huggingface.co/cstr/wav2vec2-large-xlsr-53-english-GGUF/resolve/main/wav2vec2-xlsr-en-q4_k.gguf

./build/bin/crispasr -m wav2vec2-en-q4k.gguf -f samples/jfk.wav
# and so my fellow americans ask not what your country can do for you ask what you can do for your country

# German
curl -L -o wav2vec2-de-q4k.gguf \
    https://huggingface.co/cstr/wav2vec2-large-xlsr-53-german-GGUF/resolve/main/wav2vec2-xlsr-de-q4_k.gguf

./build/bin/crispasr -m wav2vec2-de-q4k.gguf -f audio.de.wav

# Convert any HuggingFace Wav2Vec2ForCTC model:
python models/convert-wav2vec2-to-gguf.py \
    --model-dir jonatasgrosman/wav2vec2-large-xlsr-53-german \
    --output wav2vec2-de.gguf --dtype f32
# Then optionally quantize:
./build/bin/crispasr-quantize wav2vec2-de.gguf wav2vec2-de-q4k.gguf q4_k

---

Streaming, TTS, and HTTP server

CrispASR has three feature areas that warrant their own docs pages:

• Streaming & live transcription — --stream, --mic, --live, sliding-window chunking, per-token confidence, and live translation (--live-translate: transcript plus its translation, sentence by sentence, with m2m100 or a translation LLM such as Hy-MT2).
• Text-to-Speech (TTS) — Kokoro (multilingual, smallest), Qwen3-TTS (highest fidelity, voice cloning), VibeVoice (lowest-latency streaming), Orpheus (3 B Llama + SNAC), Chatterbox (flow-matching + HiFT vocoder, German via Kartoffelbox), IndexTTS, VoxCPM2, and CosyVoice3 (9 langs + 18 zh dialects; baked-voice bank + arbitrary-WAV cloning). Voice packs, language routing, and qwen3-tts environment switches. All TTS output is watermarked; post-embed verification warns if confidence is low. Use --detect-watermark file.wav to check any WAV for AI watermarks.
• Server mode (HTTP API) — persistent model, OpenAI-compatible /v1/audio/transcriptions (ASR) and /v1/audio/speech + /v1/voices (TTS, automatic on any loaded CAP_TTS backend), per-request voice + speed + instructions, CORS, long-form sentence chunking, API keys, Docker Compose, prebuilt CUDA images.
• Concurrency, parallelism & scaling — one transcription already uses multiple cores; the server accepts requests concurrently but serializes inference on one model by default; --server-workers N runs N model instances so pure-ASR requests run concurrently; and for bulk/throughput workloads, process-level fan-out (xargs -P / GNU parallel) or N replicas behind a load balancer. Also covers what is not supported (batched multi-stream inference, PagedAttention) and why.

Quickest taste of each:

# Streaming from microphone
crispasr --mic -m model.gguf

# Microphone, transcribed and translated live (German -> English)
crispasr --live-translate -l de --tr-tl en -m auto --backend parakeet --translate-model hy-mt2

# TTS via auto-downloaded VibeVoice (~636 MB on first run)
crispasr --backend vibevoice-tts -m auto --tts "Hello world" --tts-output hello.wav

# CosyVoice3 on GPU; companions auto-download beside the LLM
crispasr --backend cosyvoice3-tts -m auto --tts "Hello world" --tts-output cosy.wav

# CosyVoice3 fast mode: 5 flow steps instead of the quality-default 10
COSYVOICE3_FLOW_STEPS=5 crispasr --backend cosyvoice3-tts -m auto \
  --tts "Hello world" --tts-output cosy-fast.wav

# Persistent HTTP server, OpenAI-compatible
crispasr --server -m model.gguf --port 8080
curl -F "[email protected]" http://localhost:8080/v1/audio/transcriptions

# TTS over HTTP — load a TTS backend, hit /v1/audio/speech
crispasr --server --backend qwen3-tts-customvoice -m auto --voice-dir ./voices --port 8080
curl http://localhost:8080/v1/audio/speech \
  -H 'Content-Type: application/json' \
  -d '{"input":"Hello world","voice":"vivian"}' -o out.wav

CosyVoice3 uses batched classifier-free guidance and request-sized KV caching by default. Baked voices load only the LLM, flow, HiFT, and voice bank; the larger S3 tokenizer and CAMPPlus companions load lazily when a .wav cloning voice is first requested.

---

CLI reference

Common flags:

crispasr -m auto --backend parakeet -f audio.wav --vad -osrt --split-on-punct

Flag	Meaning
-m FNAME / --backend NAME	Model path (or auto) and forced backend
-f FNAME	Input audio (repeatable; positional accepted)
--vad	Silero VAD chunking — strongly recommended for multi-minute audio
-osrt / -ovtt / -otxt / -oj / -ojf	Output formats (also -ocsv, -olrc)
-am FNAME	CTC aligner GGUF for word-level timestamps on LLM backends
--align-only	Standalone forced alignment: text/.srt + audio → timestamped SRT/JSON (no ASR needed); .srt input keeps its cues and gets re-timed (--align-granularity auto|word|segment)
-tp F / -bs N	Sampling temperature / beam search width
-n N / --frequency-penalty F	Generated-token cap / opt-in repeated-token penalty for supported autoregressive ASR backends
-l auto / --detect-language	LID pre-step for backends without native lang detect
--hotwords "A,B,C"	Contextual biasing — boost named terms during CTC/TDT decode or LLM prompt
-ck N	Fallback chunk size when VAD is off (default 30 s)
--list-backends	Print the capability matrix and exit

See docs/cli.md for the full reference: every flag, VAD details, CTC alignment workflow, output JSON layout, the auto-download registry, and supported audio formats. See docs/bindings.md for Python / Rust / Dart / Go / Java / JavaScript / Ruby / mobile.

---

Architecture, contributing, regression matrix

CrispASR is structured as a stable C-ABI in src/ (every algorithm: VAD, diarize, LID, alignment, cache, registry) consumed by all language wrappers, with thin presentation layers in examples/cli/. Per-model runtimes live in src/{whisper,parakeet,canary,...}.cpp, sharing primitives from src/core/ (mel, ffn, attention, GGUF loader, FastConformer / Conformer / Granite-LLM blocks, etc.).

• docs/architecture.md — full layered layout, file-by-file tour of src/ and examples/cli/, per-backend internals table, regression discipline.
• docs/contributing.md — adding a new backend in five files, clang-format-18 setup, the crispasr-diff PyTorch-ground-truth workflow, and the TTS audio-cosine-vs-reference regression target.
• docs/regression-matrix.md — tools/test-all-backends.py capability tiers, cache modes (keep / ephemeral), --skip-missing for CI.
• PLAN.md — current work and ordered follow-ups. Completed investigations are removed from this file rather than left as stale tasks.
• HISTORY.md — shipped work and the archived reasoning behind earlier roadmap decisions.

Shared libraries (cross-repo with CrispEmbed):

• crisp_audio/ — Whisper-shape audio encoder (Conv-stem + Transformer)
• crisp_punc/ — punctuation restoration (FireRedPunc + PCS)
• crisp_lid/ — text-based language identification (fastText + CLD3)
• crisp_truecase/ — truecasing (statistical + CRF + BiLSTM)

Both are self-contained static libraries with CMakeLists.txt. CrispEmbed links them via add_subdirectory(../CrispASR/crisp_*/); CrispASR uses them directly. If the shared dir is absent, both repos fall back to local copies of the source files.

For benchmarks see PERFORMANCE.md; for the session-by-session port log and the bug-class lessons, see LEARNINGS.md.

---

Quantize models

build/bin/crispasr-quantize is a single, model-agnostic GGUF re-quantization tool that works across all supported model families (Whisper, Parakeet, Canary, Cohere, Voxtral, Qwen3, Granite, Wav2Vec2, MiMo-ASR, GLM-ASR, Moonshine, VibeVoice, Kokoro, Qwen3-TTS, …):

./build/bin/crispasr-quantize input.gguf output.gguf q4_k

See docs/quantize.md for the full guide: supported quant types, K-quant alignment fallback, recommended quant per backend, and worked examples for each architecture.

---

GPU backend selection

All backends use ggml_backend_init_best() which automatically picks the highest-priority compiled backend: CUDA > Metal > Vulkan > CPU. To force a specific backend:

# Force Vulkan even when CUDA is available
crispasr --gpu-backend vulkan -m model.gguf -f audio.wav

# Pin a specific GPU (useful on Vulkan systems with iGPU + dGPU)
crispasr --gpu-backend vulkan -dev 1 -m model.gguf -f audio.wav

# Force CPU (useful for benchmarking)
crispasr -ng -m model.gguf -f audio.wav

# CUDA unified memory (swap to RAM when VRAM exhausted)
GGML_CUDA_ENABLE_UNIFIED_MEMORY=1 crispasr -m model.gguf -f audio.wav

Build flags: -DGGML_CUDA=ON, -DGGML_METAL=ON, -DGGML_VULKAN=ON, -DGGML_CANN=ON.

Notes:

• --gpu-backend vulkan selects the Vulkan backend, but it does not choose which physical GPU to use. Use -dev N to select the Vulkan device index.
• On some Windows laptops, Vulkan device 0 is the Intel iGPU and the NVIDIA GPU is 1. If Vulkan looks unexpectedly slow, rerun with -dev 1.
• The Windows convenience script build-vulkan.bat creates a separate Vulkan-capable binary at build-vulkan\bin\crispasr.exe.

---

Debugging & profiling

For most backends, -v / --verbose surfaces per-stage timings and device picks. For headless / library use (where the CLI flag isn't plumbed through), set CRISPASR_VERBOSE=1 instead.

# Per-stage timing breakdown (mel / encoder / prefill / decode):
crispasr -v --backend gemma4-e2b -m model.gguf -f audio.wav
# gemma4_e2b: mel 128x1099 (17.2 ms)
# gemma4_e2b: encoder done: 1536x275 (719.0 ms)
# gemma4_e2b: prefill done, first_token=3133 (1464.0 ms)
# gemma4_e2b: decoded 25 tokens (7748.3 ms total)
# crispasr: transcribed 11.0s audio in 7.75s (1.4x realtime)

# Hugging Face access for gated models (Voxtral, Gemma4-E2B, …):
HF_TOKEN=hf_xxx crispasr -m auto --backend gemma4-e2b -f audio.wav

The server has its own auth env: CRISPASR_API_KEYS (see Server mode).

These are useful when porting a new backend or chasing a regression. The *_BENCH=1 toggles emit per-stage timings even without -v; the *_DEBUG=1 toggles emit per-step diagnostic prints; the *_DUMP_DIR= paths write per-stage F32 tensors for diff-testing against a PyTorch reference (see Debug a new backend against PyTorch ground truth).

Env var	Purpose
CRISPASR_VERBOSE=1	Forces verbose mode for any backend (parallel to the -v flag).
CRISPASR_SCHED_PROFILE=1	Forces per-node scheduler splits and reports wall time by op, source dtype and shape for Canary CTC, Cohere, FireRed-ASR, Granite Speech, Moonshine, Moonshine Streaming and Paraformer. Use relative shares; the forced splits add dispatch overhead.
CRISPASR_DUMP_DIR=path/	Generic per-stage F32 tensor dump for the crispasr-diff harness.
GEMMA4_E2B_BENCH=1	Per-stage timings for the Gemma-4-E2B backend.
COHERE_BENCH=1 / COHERE_DEBUG=1	Cohere transcribe per-stage timings / per-step diagnostics.
COHERE_PROF=1	Cohere graph-level profiling (per-op timings).
COHERE_THREADS=N	Override thread count for the Cohere backend.
COHERE_DEVICE=cpu|cuda|metal|vulkan	Force the Cohere backend onto a specific device.
COHERE_DUMP_ATTN=path/	Dump attention activations for Cohere (used by the diff harness).
FIRERED_BENCH=1	Per-stage timings for the FireRedASR backend.
FIREREDPUNC_DEBUG=1	Per-step diagnostics for the FireRed punctuation post-step.
MOONSHINE_STREAMING_BENCH=1	Per-stage timings for moonshine-streaming.
MOONSHINE_ONNX_BENCH=1	Per-graph CPU wall time for Moonshine ONNX frontend, encoder, adapter, cross-KV and decoder.
OMNIASR_BENCH=1 / OMNIASR_DEBUG=1 / OMNIASR_DUMP_DIR=	OmniASR per-stage timings, diagnostics, and stage dumps.
PARAKEET_DEBUG=1	Parakeet TDT per-step diagnostics (joint network, blank-id sanity).
QWEN3_TTS_BENCH=1 / QWEN3_TTS_DEBUG=1 / QWEN3_TTS_DUMP_DIR=	Qwen3-TTS per-stage timings, diagnostics, and stage dumps.
VIBEVOICE_BENCH=1 / VIBEVOICE_DEBUG=1 /

HF_TOKEN and HUGGING_FACE_HUB_TOKEN are both honoured for gated-model downloads (in that order).

---

Credits

• whisper.cpp — the original ggml inference engine and Whisper runtime this fork is built on
• ggml — the tensor library everything runs on
• NVIDIA NeMo — parakeet-tdt-{0.6b-v2,0.6b-v3,1.1b}, parakeet-tdt_ctc-{110m,1.1b,0.6b-ja}, parakeet-ctc-{0.6b,1.1b}, canary-{1b-v2,180m-flash}, canary-ctc aligner, and the FastConformer-CTC family (stt_en_fastconformer_ctc_{large,xlarge,xxlarge} plus CTC branches of the stt_*_fastconformer_hybrid_large[_pc] fleet: en-pc, de, es, fr, it, nl, pl, ru, ua, hr, be, ar, fa, ka, hy, uz, kk-ru — all usable both as ASR backends and as compact ~82 MB -am forced aligners)
• Cohere — cohere-transcribe-03-2026
• Qwen team (Alibaba) — Qwen3-ASR-0.6B, Qwen3-ASR-1.7B, Qwen3-ForcedAligner-0.6B
• Mistral AI — Voxtral Mini 3B and 4B Realtime
• IBM Granite team — Granite Speech 3.2-8b, 3.3-2b, 3.3-8b, 4.0-1b
• Meta / wav2vec2 — wav2vec2 CTC models (XLSR-53 English, German, multilingual via any Wav2Vec2ForCTC checkpoint)
• sherpa-onnx — optional diarization via subprocess (ONNX models)
• Silero — VAD (native GGUF) and language identification (native GGUF, 95 languages)
• pyannote — speaker diarization segmentation (native GGUF port)
• miniaudio + stb_vorbis + libopus/opusfile — embedded/linked audio decoders (WAV/MP3/FLAC/AIFF/OGG/Opus, no ffmpeg; AAC/M4A/ALAC via Apple AudioToolbox)
• glint (MIT) — in-tree clean-room codec suite: MP3/AAC-LC/Ogg-Opus output encoders (TTS //) and cross-platform ADTS AAC-LC + Ogg Opus input decoders (no libopus needed)

---

License

Same as upstream whisper.cpp: MIT.

Per-model weights are covered by their respective HuggingFace model licenses (see Supported backends). The crispasr binary itself links model runtimes that are mostly permissively licensed (MIT / Apache-2.0 / CC-BY-4.0 for weights).