EchoBuddy — Voice-to-Voice AI Chatbot

A voice chatbot with two pipelines behind one UI:

• Realtime (WebSocket): live mic streaming, server-side voice-activity detection, streamed text and audio replies, tool calling and document search.
• Batch (Temporal.io): record a clip, get a full reply; fault-tolerant, with retries.
• Text chat: type a message, get a text and spoken reply.

Stack: Deepgram (speech-to-text) · Groq / OpenRouter (LLM) · Cartesia (text-to-speech) · FastAPI · Temporal · SQLite · ChromaDB (RAG) · Langfuse (tracing).

Running locally

cp .env.example .env            # then fill in the API keys
uv sync

docker compose up -d            # Terminal 1: Temporal server (batch mode only)
uv run python -m temporal.worker                          # Terminal 2: Temporal worker (batch mode only)
uv run uvicorn realtime.server:app --reload --port 8000   # Terminal 3: API + UI

# open http://localhost:8000

The realtime and text-chat modes need only the API server. Temporal is required just for the 🎤 batch mode.

Configuration — nothing is hardcoded

config/settings.py is the single source of truth. Every value below is a field there and can be overridden by an environment variable of the same name in UPPER_CASE (see .env.example). Long prompts live in config/prompts.py.

Area	Examples
Credentials	DEEPGRAM_API_KEY, GROQ_API_KEY, CARTESIA_API_KEY, OPENROUTER_API_KEY, LANGFUSE_*
LLM	GROQ_MODEL, OPENROUTER_MODEL, SYSTEM_PROMPT, HISTORY_MAX_TURNS, LLM_*_TEMPERATURE, FORCE_TOOL_KEYWORDS
Speech	DEEPGRAM_MODEL, CARTESIA_MODEL, CARTESIA_VOICE_ID, TTS_SAMPLE_RATE
Voice detection	VAD_AGGRESSIVENESS, VAD_SILENCE_TIMEOUT_MS, VAD_MIN_SPEECH_RMS
Tools	WEB_SEARCH_MAX_RESULTS, WEATHER_URL_TEMPLATE, CALCULATOR_MAX_EXPONENT
Documents (RAG)	EMBEDDING_MODEL, RAG_CHUNK_SIZE, RAG_TOP_K, RAG_MAX_DISTANCE, MAX_UPLOAD_MB
Server	LOG_LEVEL, CORS_ORIGINS, ALLOWED_AUDIO_EXTENSIONS
Access control	API_TOKEN, RATE_LIMIT_REQUESTS, RATE_LIMIT_WINDOW_SECONDS, TRUST_PROXY_HEADERS
Temporal	TEMPORAL_ADDRESS, TEMPORAL_TASK_QUEUE, ACTIVITY_* retry and timeout tuning

The old GEMINI_* variable names are still accepted as aliases for the Groq settings.

Architecture

config/                      Settings (settings.py), prompts (prompts.py), Langfuse setup (tracing.py)
core/                        Logic shared by BOTH pipelines
├── prompting.py             system prompt + the user's document excerpts + history, tool-forcing heuristic
├── tool_loop.py             the two tool-calling steps (decide, run) against any OpenAI-style client
├── tools.py                 tool registry: web search, weather, calculator, date/time
└── assets.py                `assets/<name>` audio references (no filesystem paths leak into API/DB)
database/
├── connection.py            async SQLite connection + schema bootstrap
├── repository.py            the ONLY place that issues SQL (used by both pipelines)
└── schema.sql               users / sessions / messages
realtime/                    Realtime + text chat
├── server.py                FastAPI app setup (middleware, mounts, startup, background tasks)
├── routes/
│   ├── chat.py              POST /api/voice-chat (Temporal), POST /api/text-chat
│   ├── ws.py                WS /ws (realtime voice)
│   ├── documents.py         /api/upload-doc, /api/docs   (per-user)
│   ├── auth.py · health.py  /api/auth/check, /health
├── security.py              access token, rate limiting, user-id validation
├── session.py               per-connection turn-taking (VAD → STT → LLM → TTS)
├── llm_stream.py            streaming reply (Groq) on top of core/
├── stt.py · tts_stream.py   Deepgram / Cartesia
├── audio.py · cleanup.py    PCM/WAV helpers · old-audio cleanup
rag/                         ingestor.py (PDF/TXT → ChromaDB, per owner), retriever.py (semantic search)
temporal/                    Batch pipeline
├── workflows.py             VoiceChatbotWorkflow (sequencing only, no I/O)
├── worker.py · dto.py
└── activities/              one external dependency per file (stt, llm via OpenRouter, tts, db)
static/                      the UI: index.html, plus build.mjs + vercel.json for the Vercel deploy
tests/                       pytest suite · tests/harness: live end-to-end, load and LLM-quality checks

Dependencies point inward: routes / activities → core → config / database / rag. Both pipelines share core/, so they behave the same (tools, document search, prompts). Nothing outside database/repository.py issues SQL, and no tunable is hardcoded.

This is the beginning of the README. Read the rest at the original. Read the full README ↗