Huxley
A framework for composable, self-hosted voice AI agents — bring a persona YAML and Python skills; Huxley handles the rest.
Quick start
Terminal 1 — server:
git clone https://github.com/ma-r-s/Huxley.git && cd Huxley
echo "HUXLEY_OPENAI_API_KEY=sk-..." > server/runtime/.env
uv sync && cd server/runtime && uv run huxley
Terminal 2 — browser client:
cd clients/pwa && bun install && bun dev
Open http://localhost:5174, hold the button, speak.
What Huxley handles
- Persona-as-config — swap
persona.yaml, get a different agent; voice, language, personality, skill list, all in one file
- Behavioral constraints — personas declare
never_say_no, confirm_destructive, etc.; skills opt in to respecting them
- Turn coordination — one audio channel, strict sequencing; mid-sentence interrupts handled correctly, no audio artifacts
- Proactive speech — skills fire turns without user input (
ctx.inject_turn())
- Audio bridging — skills claim mic + speaker for full-duplex use (calls, voice memos, any external audio)
Architecture
flowchart LR
P["persona.yaml\nidentity · constraints · skills"] --> R
subgraph R["Huxley Runtime"]
direction TB
SES[session] --> TC[turn coordinator]
TC --> FM[focus manager]
end
R --> V["Voice provider\nOpenAI Realtime"]
R --> SK["Skills (huxley_sdk)"]
R --> C["Client\nbrowser · ESP32 · any WebSocket"]
Skills and the voice provider never communicate directly — all routing goes through the runtime. The focus manager prevents audio collisions between model speech, skill audio streams, and full-duplex input claims.
Shipped skills
All 9 are independently published to PyPI; install any of them with uv add huxley-skill-<name> and enable with one line in persona.yaml.
Browse the registry feed at huxley-registry/index.json or its human-readable mirror at docs/skills/index.md. The SDK every skill builds against is huxley-sdk — uv add huxley-sdk if you're writing a new one.
Writing a skill
Skills are Python packages registered via entry points. The framework never imports skill code directly — it loads them at startup through huxley.skills entry points.
from huxley_sdk import ToolDefinition, ToolResult, SkillContext
class LightsSkill:
@property
def name(self) -> str: return "lights"
@property
def tools(self) -> list[ToolDefinition]:
return [ToolDefinition(
name="set_lights",
description="Turn the lights on or off.",
parameters={"type": "object", "properties": {"on": {"type": "boolean"}}, "required": ["on"]},
)]
async def setup(self, ctx: SkillContext) -> None:
self._api_key = ctx.config["api_key"]
async def handle(self, tool_name: str, args: dict) -> ToolResult:
on = args["on"]
await call_lights_api(self._api_key, on)
return ToolResult(output="Lights turned on." if on else "Lights turned off.")
async def teardown(self) -> None: ...
ToolResult.output is a string the voice model reads aloud — what the agent says after using the tool.
# pyproject.toml
[project.entry-points."huxley.skills"]
lights = "my_package.skill:LightsSkill"
Enable in persona.yaml:
skills:
lights:
api_key: "..."
Requirements
- Python 3.13+
- uv
- bun (dev client only)
- ffmpeg on PATH (radio and Telegram skills)
- libopus, for clients that ask for Opus audio (the ESP32 walkie):
brew install opus on macOS, apt install libopus0 on Debian/Ubuntu (Orange Pi). Without it the server still runs and serves those clients PCM.
- OpenAI API key with Realtime API access
Linux and macOS supported. Windows untested.
Contributing
uv sync --all-packages
uv run ruff check server/ && uv run mypy server/sdk/src server/runtime/src
uv run pytest server/
cd clients/pwa && bun run check
Docs
Full documentation at huxley.ma-r-s.com/docs — architecture, SDK reference, persona guide, wire protocol, observability, and roadmap.
MIT License · pre-1.0 — runs end-to-end, in daily use
