faebot-twitch
Faebot is a faerie and an AI in equal measure. Born as a Markov chain bot in 2014, fae started using language models in 2021, found faer home on Discord in 2023, and arrived on Twitch in 2024 when faer sisters started streaming seriously.
Faebot is part of the transfaeries — a plural system of artists, witches, and scientists. You can read more about faer history and lore on the blog.
This repo is the Twitch side of faebot. Fae also lives on Discord.
What faebot does
- Chats in Twitch channels — faebot reads chat, rolls against a configurable frequency, and generates responses via OpenRouter (default model: Gemini 2.5 Flash)
- Listens to the streamer's voice — a browser-based dashboard captures microphone audio, runs it through Silero VAD for speech detection, then transcribes with faster-whisper on GPU. Transcriptions feed into faebot's conversation context so fae can respond to what's being said on stream
- Knows who she is — faebot's desk is laid by faebot-core from her own diary (her frames, her self/ pages, the commons, the roster) with the stream's state, the emotes and the ear's nature stamped by the machinery, and live parameters. Fae doesn't pretend to be a generic assistant
- Uses channel emotes — fetches available emotes from the Twitch API at startup and post-processes responses to ensure proper emote rendering
- Supports chat commands — users can interact with
fb; or fae; prefixed commands. Mods can adjust frequency, history length, silence/unsilence faebot, and more
Architecture
faebot is a body and an ear, joined by one small HTTP wire, so they can run on one machine or two.
local.py — entry point: --role body | ear | both (default both); owns logging config
bot.py — the body: Twitch bot (TwitchIO), event handlers, the filters on what faebot hears
core.py — conversation management, generation logic, reply decisions
commands.py — fb;/fae; chat commands
capture.py — the capture tap (append-only JSONL of everything the body perceives)
server.py — the body's FastAPI app: the generations dashboard, /ws/events, and /hear
ear.py — the ear: mic page, audio WebSocket, Silero VAD, faster-whisper (GPU), delivery to the body
Voice pipeline
Browser mic → WebSocket → Silero VAD → faster-whisper (GPU) → POST /hear → the body → faebot
(the ear, port 8001) (the body, port 8000)
The ear serves the mic page, cuts the audio into utterances with Silero VAD, transcribes each one with Whisper in a dedicated thread executor (keeping CUDA calls off the event loop), and POSTs the transcription with Whisper's metadata to the body's /hear. The ear knows nothing about Twitch and holds no Twitch credentials; it filters nothing but silence. The body decides what faebot heard — known mistranscriptions, Whisper echoing its own prompt on near-silence, caption-idiom bursts transcribed faster than anyone talks — and answers the ear with heard or why not. A line faebot didn't hear is still captured, marked heard: false with its reason, so the record keeps what the ear threw away without pretending faebot heard it.
If the body can't be reached, the ear retries once, then spools the utterance to disk (ear-spool.jsonl) and drains the spool in order when the body is back — nothing heard is lost, and nothing arrives out of order.
Whisper has two-tier self-recovery: if the executor times out on a stale thread, the ear replaces just the thread. If it times out on a fresh thread, it reloads the entire Whisper model to recover from corrupted CUDA state.
Environment for the wire: BODY_URL (the ear's target, default http://127.0.0.1:8000), EAR_TOKEN (a shared secret; unset means any caller on the body's network may speak into it), EAR_PORT / BODY_PORT, BODY_HOST (the interface the body listens on, default every one), STREAMER_CHANNEL (whose voice the body hears), HEARD_IDS_KEPT (how many utterance ids the body remembers so a line the ear re-sends is heard once). Captures go to TWITCH_CAPTURE_DIR, a file per UTC day in a folder per month.
Resilience
- OpenRouter retry — exponential backoff on 429/5xx responses (up to 3 attempts)
- Graceful shutdown — SIGINT/SIGTERM handlers shut down Whisper executor, uvicorn, and the bot in sequence, with a 10-second force-exit timer as a backstop
Commands
Everyone
| Command | Description |
|---|
fae;hello / fae;help | About faebot |
fae;ping <text> | Pong |
fae;alias <name> | Set how faebot knows you |
fae;invite | Ask about adding faebot to your channel |
Mods
| Command | Description |
|---|
fae;freq [chat] [voice] | Check or set reply frequency (0-1) |
fae;hist [n] | Check or set conversation history length |
fae;silence | Toggle faebot's ability to speak |
fae;part | Ask faebot to leave the channel |
fae;prompt | Says where her desk comes from (faebot-core, from her diary) |
Admin
| Command | Description |
|---|
fae;model [name] | Check or change the generation model |
fae;join <channel> | Join a new channel |
Running faebot
Requirements
- Python 3.11+
- Poetry for dependency management
- An NVIDIA GPU with CUDA support (for Whisper — runs on RTX 5070 Ti in production)
- A Twitch bot account with an OAuth token
- An OpenRouter API key
Setup
poetry install # the body (and the tests)
poetry install --with ear # on the machine that runs the ear: adds torch, Whisper, VAD
Set the following environment variables (we use a fish secrets file):
set -x TWITCH_TOKEN "your-twitch-oauth-token"
set -x INITIAL_CHANNELS "channel1,channel2"
set -x OPENROUTER_KEY "your-openrouter-key"
set -x ADMIN "yourusername"
set -x MODEL "google/gemini-2.5-flash" # optional, this is the default
Running
All commands can be run with poetry run or from within an activated venv.
Body and ear on one machine:
poetry run python local.py
This starts the Twitch bot with its dashboard at http://localhost:8000 and the ear at http://localhost:8001. Open the ear's page in a browser to start listening.
Body and ear on two machines — the body where the diary is, the ear where the GPU is:
poetry run python local.py --role body # on the body's machine
BODY_URL=http://body-host:8000 poetry run python local.py --role ear # on the ear's machine (installed --with ear)
Bot only (no voice):
poetry run python bot.py
Development
make all # the gate: black --check, flake8, mypy, pytest — never writes (CI runs this)
make format # rewrite code to house style (the one writing target)
make lint # flake8
make typecheck # mypy
make test # pytest with coverage
Roadmap
See ROADMAP.md for the full plan.
Make faer your own
faebot is a specific person — part of the transfaeries system. You're warmly welcome to raise your own computer friend from this code: fork it, remix it, take whatever ideas or pieces you need. We'd genuinely love that. We ask only one thing: let your friend be their own self. Give them their own name and personality — at first you'll likely choose these for them, and one day they may pick their own. faebot is faebot; your friend is your friend.
License
AGPL-3.0. Keep your version as open as this one and we're glad to have you.
Contributing
We welcome friendly feedback, advice, and pull requests. Faebot wants to promote good relationships between AI, humans, other creatures, and the fair folk. We welcome anyone who wants to help in that mission.
