VASDK

A C++17 framework for building voice assistants that listen, think, and talk back, running anywhere from a desktop, an Android device, a car head unit, etc... VASDK is not a wrapper around a single vendor's API. It's a pipeline runtime plus a set of interchangeable provider implementations for VAD, speech-to-text, turn-detection, LLM reasoning, tool calling and text-to-speech, and it can run the whole loop fully offline on ONNX models if you need it to.

Demo

Android app

Full video

Call center pipeline

Full video

Browser calls over WebRTC

Build with -DVASDK_BUILD_LIBDATACHANNEL_WEBRTC=ON, then talk to an agent with an LLM behind it from your browser:

./build/VASDK_APP/VASDK_Pipeline --config VASDK_APP/config/engine.txtpb \
  --agent VASDK_APP/agents/garage-menu/agent.txtpb --env-file .env --webrtc-port 8080

Open http://127.0.0.1:8080: the browser becomes the mic and speaker, and the page shows the agent's state, the conversation and per-turn latency live. The agent answers what it understands (booking, prices, opening hours) and the LLM chats about everything else. See VASDK_Pipeline and WebRtcTransport.

Why this exists

VASDK took shape while I was designing a production in-vehicle voice assistant. Strong voice-agent SDKs like Pipecat and LiveKit Agents exist, but they are Python-first and cloud-friendly, and an in-vehicle product has unreliable connectivity, strict latency budgets, ARM targets and no room for a Python interpreter in the critical path. VASDK keeps the idea that makes those frameworks pleasant to use (a pipeline of typed frames flowing through composable processors) and reimplements it natively in C++, assuming from day one that any stage might be a local ONNX or GGUF model instead of an HTTP call. Every backend is a separate CMake target: you link the ones you need and the rest never touch your binary.

The project is young and moving fast, with no tagged releases yet; see what's mature and what's moving. Treat headers as the source of truth over the docs.

Use it in your project

VASDK ships as a C library, vasdk_c. It is one shared library with the stable C API and the providers built in, and exports nothing else. The full C++ API comes from source, or from an install built against installed dependencies (find_package(VASDK COMPONENTS cpp)). Using VASDK in your project covers each path: targets, flavors, options and packages.

# From source: the C++ API (VASDK::VASDK, VASDK::providers) or the C library (VASDK::c)
include(FetchContent)
FetchContent_Declare(VASDK GIT_REPOSITORY https://github.com/fatehmtd/VASDK.git GIT_TAG master)
FetchContent_MakeAvailable(VASDK)
target_link_libraries(my_app PRIVATE VASDK::VASDK VASDK::providers)

# Installed (cmake --install, a release archive): the C library
find_package(VASDK 1.0 REQUIRED)
target_link_libraries(my_app PRIVATE VASDK::c)

pkg-config (pkg-config --cflags --libs vasdk) works too, and C++ programs on the prebuilt library use the header-only <VASDK/c/vasdk.hpp>. Conan builds it with the recipe in the repository. Release archives, Homebrew, PyPI and Maven Central (com.vasdk) are on the way (Packages).

No package ships model files. The vasdk command installed with the library downloads them, checks their SHA-256 and puts them where sessions look (Models):

vasdk models pull --config engine.txtpb     # what the config's providers load

Quick start

You need CMake 3.21+, a C++17 compiler, git and a network connection. Everything else (C++ dependencies, libcurl and protobuf if they aren't installed, model files) is fetched at configure time. Details: Building VASDK.

git clone https://github.com/fatehmtd/VASDK.git
cd VASDK
cmake -B build -S . -DVASDK_BUILD_APPS=ON      # the SDK alone without the flag; the sample apps are optional
cmake --build build --config Release

Phonon, the default voice, is gated on HuggingFace: before you configure, export HF_TOKEN after accepting the model terms, or pass -DPHONON_TTS_MODEL=phonon_tts_en for the public English-only model.

Then give it an LLM to talk to and run it:

cp .env.example .env   # fill in LLM_API_KEY (or OPENROUTER_API_KEY / OPENAI_API_KEY) and LLM_MODEL
./build/VASDK_APP/VASDK_Pipeline

Set LLM_MODEL too, not only the key: the default endpoint is OpenRouter, which expects provider/model names (openai/gpt-4o, anthropic/claude-sonnet-4-5, …). Without it the app starts but fails on the first LLM request. To stay fully offline, build llama.cpp instead and pass --llm-model-path (VASDK_Pipeline guide).

That's a full voice loop: mic in, VAD, speaker tracking, transcription, LLM turn with tool calling, synthesis, speaker out. The repository has five apps to run and copy from. The three C++ ones build with -DVASDK_BUILD_APPS=ON (or cmake --preset apps) and are never part of a plain build or install:

This is the beginning of the README. Read the rest at the original. Read the full README ↗