🦜 VieNeu-TTS

VieNeu-TTS is the next generation of on-device Vietnamese TTS: 10,000+ hours of bilingual training, instant voice cloning, a dedicated Podcast/Conversation mode, and extremely fast — real-time streaming with first audio in ~115 ms, RTF ≈ 0.01–0.02 batched on a consumer GPU (RTX 3060), ~2× real-time on a plain CPU (benchmarks).

[!IMPORTANT] 🦜 VieNeu-TTS v4 — available on vieneu.io

VieNeu-TTS v4 delivers near-original voice cloning fidelity, allowing a short reference clip to be reproduced with very high speaker similarity.

Due to the strength of its voice-cloning capabilities and the potential for misuse, v4 is proprietary and will not be open-sourced. It is available exclusively through the VieNeu API / vieneu.io.

VieNeu-TTS v3 Turbo remains the latest open-source version available in this repository. Future open-source releases, including potential v3.x updates, may also be published here.

[!NOTE] 🦜 VieNeu-TTS v3 Turbo is officially released! A brand-new architecture designed and trained from scratch by Phạm Nguyễn Ngọc Bảo (codec: MOSS-Audio-Tokenizer-Nano; phonemizer: sea-g2p):

• 48 kHz high-fidelity audio (up from 24 kHz).
• 25 built-in preset voices (10 editors' picks) — stable and consistent, no reference clip needed.
• Natural reading style everywhere — the style follows the reference voice (the style argument is deprecated and ignored).
• Emotion / non-verbal cues (experimental): drop [cười], [thở dài], [hắng giọng] straight into the text.
• Batched generation (batch size up to 32), including a multi-speaker Conversation mode that batches the whole script regardless of speaker.
• Instant voice cloning from a 3–8s clip, with automatic reference denoising.
• Real-time streaming + OpenAI-compatible API + Docker — POST /v1/audio/speech drop-in for the OpenAI SDK / Pipecat / LiveKit; first audio in ~115 ms and 16 concurrent streams under 200 ms on a single RTX 3060 (32 max), CPU-only streaming too. See §3, §4 Benchmarks and docs/streaming.md.
• LoRA fine-tuning — train your own voice or reading style on one consumer GPU (§5).

Try it in the Web UI (backbone "VieNeu-TTS-v3-Turbo") or the SDK (Vieneu(mode="v3turbo"), the default).

[!TIP] Dubbing, Lecture, Tiktok Reel and Audiobook features are exclusively available in the official VieNeu App. This GitHub repository only provides a simplified Gradio demo interface and the core SDK for developers.

📌 Table of Contents

1. 🦜 Installation & Web UI
2. 📦 Using the Python SDK
3. 🐳 API Server & Docker — OpenAI-compatible streaming API (v3 Turbo) · legacy v2 server
4. 📊 Benchmarks — every speed / latency number in one place (CPU vs GPU, batch, streaming, Nano)
5. 🎓 Fine-tuning (LoRA)
6. 🔬 Model Overview
7. 🚀 Roadmap
8. 🤝 Support & Contact
9. 📑 Citation

---

🦜 1. Installation & Web UI

[!TIP] On Windows? The fastest way to get started is the standalone installer at vieneu.io/#/download — no need to install uv or clone the repo manually. macOS: a similar installer is coming in an upcoming release; for now, please use the uv sync steps below. Docker? Skip the steps below: --profile api-gpu / api-cpu (OpenAI-compatible streaming API) — see §3 API Server & Docker.

Setup with uv (Recommended)

uv is the fastest way to manage dependencies.

# Windows:
powershell -c "irm https://astral.sh/uv/install.ps1 | iex"

# Linux/macOS:
curl -LsSf https://astral.sh/uv/install.sh | sh

1. Clone the Repo:

   git clone https://github.com/pnnbao97/VieNeu-TTS.git
   cd VieNeu-TTS
   

2. Install Dependencies:

   📊 Which one? CPU ≈ RTF 0.5 (2× real-time, one stream) · GPU ≈ RTF 0.02 batched (~50× real-time) and 16 real-time streams — all measurements in §4 Benchmarks.

   • Option 1: CPU & macOS (minimal, torch-free) — RTF ≈ 0.5, no GPU needed

     uv sync
     

   • Option 2: GPU — v3 Turbo on GPU (PyTorch) — RTF ≈ 0.02 batched, 16 real-time streams

     uv sync --extra cuda
     

3. Start the Web UI:

   uv run vieneu-web
   

   Access the UI at http://127.0.0.1:7860.

---

📦 2. Using the Python SDK (vieneu)

The vieneu SDK defaults to VieNeu-TTS v3 Turbo (48 kHz). The minimal install is torch-free: on CPU everything runs on ONNX Runtime (PyTorch is never imported), and on a CUDA machine it auto-switches to the PyTorch engine — where inference is batched automatically (same API, no code change).

This is the beginning of the README. Read the rest at the original. Read the full README ↗