TARS

A household voice assistant you can take apart: say "hey TARS", ask something, and it answers in TARS's voice. It runs on a Raspberry Pi 4 (or 5) with a USB speakerphone, or on a Mac while you work on it.

See how it works, from wake word to answer, and everything it does →

At a glance

Part	Result
Wake word	Two stages, on the device: 97% of held-out voices in quiet, 97% of the owner's (never heard), 1 false answer in an hour of TV
Response time	1.2-1.5 s from you stopping to the first sound, down from about 3.5 s
Quick model	Qwen on Groq, then Cerebras, hedged at 0.5 s; 149/156 on the capability benchmark at low reasoning
Whole conversations	48 scenarios played through the real brain, hand-offs followed through: 143/145 runs right on the 29 measured so far (from 76/87); run on request from a pull request comment
Self-learning	Retrains its wake models from the household's use, installs them only if better
Runs on	Raspberry Pi 4 or 5 with a USB speakerphone (a Pi 4 with 8 GB and an Anker PowerConf here), or a Mac

Highlights

• Nothing leaves the house before the wake word's double-check agrees (wake word).
• Answers are drafted before you finish, and thrown away if you keep talking (response time).
• A quick model that hands off: web questions to OpenAI, hard ones to a thinking model (models).
• Evaluated as whole conversations: hand-offs followed through, corrections, failures mid-conversation, every turn checked, with costs estimated and capped (evals).
• Reminders, timers and messages that repeat until someone says "got it" (reminders).
• Works the house through Home Assistant: locks, switches, lights, covers, the AC, the TV. A change is "done" only when the device reports it, nothing is done by a draft, and the door unlocks only for a trusted voice that asked. An agent plays things on an Android TV, checking the screen step by step; anything else is a plugin (the house).
• Knows who's home from whose phone is on the home Wi-Fi, asking the routers themselves: no app on anyone's phone. A reminder for someone who's out goes to their Telegram chat, and a message waiting for someone is said once they're back (who's home).
• Knows who's talking, locally and in parallel, so it adds no latency (self-learning).
• Every paid service is optional (what runs instead).

How a request is answered:

flowchart LR
    mic[Mic] --> wake[Wake word] --> stt[Speech to text] --> brain[Quick brain] --> voice[Voice] --> spk[Speaker]
    brain -.-> handoff[Hand-offs]

The wake word runs on the device. Speech to text is Deepgram's Flux, the voice is Cartesia's Sonic, the quick brain is Qwen on Groq or Cerebras, and hand-offs go to OpenAI for web search and hard questions.

How it learns:

flowchart LR
    wake[Wake word] --> kept[Wakes kept] --> labels[Labelled] --> train[New pair trained] --> test[Tested] -->|better| wake

Quick start

Needs uv. On Linux (a Pi included), first sudo apt install libportaudio2 libatomic1 alsa-utils.

git clone https://github.com/alon-pilosoph/tars.git && cd tars
uv sync --all-extras                   # every extra: echo cancellation, Claude, Android TV, the IR blaster, S3 backup
cp .env.example .env                   # then your API keys: OpenAI, Deepgram, Cartesia, Groq, Cerebras, or some
uv run voice-assistant --check         # keys, devices, models, services, voice and mic: what's wrong, what to do
uv run voice-assistant --mic-test      # loudness, speech and wake-word meters; no API key needed
uv run voice-assistant                 # say "hey TARS", then ask something
uv run voice-assistant --web           # in a second terminal: the web UI at http://127.0.0.1:8080

Also --ptt (press Enter instead of the wake word), --text (type, hear answers) and --list-devices. On a Pi, deploy/install-pi.sh does it all (at home). The web UI has no accounts: keep it on your network.

Configure

Everything is in config.toml: audio devices by name, wake sensitivity, turn-taking, each stage's model. What's this machine's own (the speakerphone's name, who's on Telegram) goes in config.local.toml next to it: never committed, its settings replace config.toml's, so git pull never meets a local edit. Every key in .env is optional, as long as something can hear, think and speak; TARS says at startup what it does without the missing ones (running on the keys there are).

The persona is TARS from Interstellar: for its speaker-in-a-metal-box sound, set [tts] effect = "tars"; for a plain assistant, edit the system prompt.

Docs

Doc	What's in it
Hearing "hey TARS"	the two-stage wake word, its training and evaluation
Training the models	the training pipeline, from public data to the household's pair
Response time	where the time goes, and the steps from 3.5 s
Choosing the quick model	the capability benchmark, every model tried, routers
Evaluating whole conversations	scenario evals: hand-offs followed through, corrections, failures, and what they found
How TARS fits together	the pipeline, failures, missing keys, privacy, the code map
Self-learning TARS	what's kept, automatic labels, voices, retraining
Reminders, timers and messages	setting, saying and acknowledging them
The web UI	pages, the Refresh rule, the API, checks
TARS on Telegram	setting it up, what it does, where reminders go, what Telegram gets

Development

uv run pytest                                # offline tests: no devices or API keys
uv run ruff check src tests tools training evals examples && uv run ruff format src tests tools training evals examples
uv run python -m evals --estimate            # what the scenario evals would cost; without it they run (paid APIs)
cd webui && npm install
npm test && npm run lint && npm run format   # the web UI's unit tests, lint and formatting
npm run build                                # rebuilds the committed page in src/voice_assistant/webui_static/
npm run e2e && npm run visual                # against a real backend, and the screenshot baselines (macOS)
npm run site                                 # the site (How it works, Features), from site/ into site/dist/

CI runs the tests with every extra installed, lint and format checks, a check that the committed web build is current, and an end-to-end run; the site deploys only after CI passes on main.

License

The code is MIT (LICENSE). The "hey TARS" models in models/generic/ are CC BY-NC-SA 4.0 (models/LICENSE.md), since some of their training data is non-commercial only; every source and its terms are in ATTRIBUTION.md.