End-to-end speech processing toolkit
Documentation ·
Installation ·
Recipes ·
Model Zoo ·
Notebooks ·
Discord
ESPnet is an end-to-end speech processing toolkit built on PyTorch. It covers speech recognition, text-to-speech, speech translation, speech enhancement, speaker diarization, spoken language understanding, spoken language identification, singing voice synthesis, speech language models, and more — with Kaldi-style reproducible recipes from data preparation to evaluation, and hundreds of pretrained models on Hugging Face.
What's new
- ESPnet 202611 — audio quality assessment (
espnet2.bin.aqa_inference, the ARECHO metric model and the URGENT24 recipe), so VERSA can install its supporting models from PyPI; Bagpiper on the command line (espnet describe, espnet render) and in the MCP server; scaled dot-product attention through every ASR encoder, relative-position attention included; F5-TTS; hydra-core and omegaconf are now core dependencies, and ctc-segmentation comes from upstream 1.7.5.
- ESPnet 202610.post2 — the command line has one name a task and two more of them:
espnet phonemize reads the phones with POWSM, espnet align lines text up with the audio it was said in, and asr and tts became transcribe and synthesize (the old names still work); models published before June 2025 load again, after init: chainer was removed from the toolkit.
- ESPnet 202610.post1 — the command line grows
espnet demo (the OWSM browser demo) and --live (the microphone, transcribed as you speak); one Speech2Text now loads either kind of OWSM checkpoint, with best_path() for CTC decoding without a search; espnet/espnet:inference-cpu-latest and -gpu-latest run a published model with nothing installed; three more demo Spaces (TTS, enhancement, speaker verification).
- ESPnet 202610 — one-line inference from the command line (
pip install espnet && espnet asr audio.wav), two OWSM v4 demos as Hugging Face Spaces, a core install without the training stack (training is espnet[train]), batched beam search, PyTorch 2.11-2.14.
- ESPnet 202609 — ESPnet3 complete on
egs3/librispeech_100 at ESPnet2 parity, CI rebuilt on a prebuilt image (compute per run halved), OpenBEATs pretraining, ten new recipes (ASR, TTS, SER, ST, audio SSL), Python 3.12-3.13.
- ESPnet 202604 — Docker-based CI, PyTorch 2.9.1 support, FastSpeech2 inference ~1.9x faster at batch 8, new recipes (Kinyarwanda, Emilia, kosp2e).
- ESPnet 202511 — parallel-processing primitives, refactored inference and evaluation pipeline, expanded SpeechLM support.
- ESPnet 202509 — Python 3.9-3.13, Debian 12 CI, the LID subsystem completed, multi-optimizer training ( / ).
Full history: Releases.
Install
# Install PyTorch first: https://pytorch.org/get-started/locally/
pip install espnet # run pretrained models
pip install "espnet[train]" # also train them (Lightning, TensorBoard, W&B, Dask)
pip install "espnet[all]" # training plus every task extra (except sds)
pip install git+https://github.com/espnet/espnet # latest master
This is the beginning of the README. Read the rest at the original. Read the full README ↗
