Zeno
A local spatial computing application built with Next.js 15, React 19, Three.js, React Three Fiber and MediaPipe. See PROGRESS.md for the exact implementation and acceptance status of all seven phases.
Run
npm install
node scripts/setup-tracking.mjs
npm run dev -- --port 3001
Open http://127.0.0.1:3001/. For a production preview, run npm run build followed by npm start -- --port 3001. Node 22+ is recommended. The development disk cache is disabled because OneDrive may lock its files on Windows.
Open Zeno as an app (Windows, no terminal)
- Download the repository (Code → Download ZIP) and extract it, or keep your existing Zeno folder.
- Double-click Start ZENO.cmd. The first run installs Node.js if needed (via winget), installs components and builds Zeno; this takes a few minutes once. Later starts take seconds and only rebuild after updates.
- Zeno opens in its own app window (Edge or Chrome app mode, no tabs or address bar). The launcher also adds a Zeno shortcut to your Desktop and Start menu, and starts the Ollama app if it is installed.
The engine keeps running quietly in the background on 127.0.0.1:3001; double-click Stop ZENO.cmd to stop it. Its log is in %LOCALAPPDATA%\Zeno\engine.log.
Zeno is also an installable web app: in the app window or browser, use INSTALL APP (top right) or the install icon in the address bar to pin it to the taskbar and Start menu. If the engine is not running when you open it, the window shows a reconnect screen and opens Zeno by itself once the engine is up. The AI brain only works where Ollama runs, so use Zeno from your own computer rather than a hosted copy when you want local AI.
If Ollama is running without a model, Ask Zeno shows a Download button that fetches the model (with progress) without a terminal.
AI: local or cloud
Copy .env.example to .env.local and choose a provider. Never commit .env.local or paste credentials into the assistant UI.
- Ollama (default): Install Ollama, then run
ollama pull qwen3:1.7b. Set AI_PROVIDER=ollama and OLLAMA_MODEL=qwen3:1.7b in .env.local, and restart Zeno. Keep Ollama running while using AI. Requests stay on loopback; no API key is needed. The model download is approximately 1.4 GB and is separate from this repository. This compact model suits basic local assistance; response quality and speed differ from larger cloud models. You can select another installed chat/tool model with OLLAMA_MODEL.
- Zeno brain (own, fully local): set
AI_PROVIDER=zeno and run Zeno's Python brain (brain/, see brain/CLAUDE.md for setup and models) with python -m zeno from the brain folder; the Windows launcher starts it for you when AI_PROVIDER=zeno. The web app reads the brain's token from brain/data/api-token (override with ZENO_API_TOKEN or ZENO_API_TOKEN_FILE) and talks to http://127.0.0.1:8765 (ZENO_BRAIN_URL, loopback only). The brain keeps the conversation and long-term memory itself and can use tools; anything that changes something (saving a memory, running one of your scripts, a device command) shows Approve / Deny in Ask Zeno first.
- Gemini:
AI_PROVIDER=gemini, plus GEMINI_API_KEY. The default model is gemini-3.8-flash; override GEMINI_MODEL for your account. Requests include recent conversation and the displayed module context.
Use Ask Zeno to type or enable voice. Direct module, orbit and transformation commands work without either provider. Conversation history is memory-only. Microphone and speech output are opt-in. Browser recognition may use its vendor's servers; native speech synthesis varies by OS and is not routed through spatial audio. Escape stops active AI input/output.
Speed. Zeno keeps the Ollama model loaded (OLLAMA_KEEP_ALIVE, default 30m), warms it when the app opens, sends only the recent conversation, and keeps the system prompt identical between questions so Ollama reuses its prompt cache. Answers are kept short unless you ask for depth. The first answer after starting your PC is still slower while the model loads.
Voice. With voice on, Zeno speaks with a local neural voice (Kokoro, am_michael by default; change it with NEXUS_VOICE_ID). The first time, it downloads about 90 MB into .nexus-cache/models, so it needs internet once; until then, and whenever it is unavailable, the browser's best natural voice is used. In Ask Zeno, Character picks Ultron-style synthetic (an original deep, metallic, slightly doubled character voice, not a copy of any actor) or Natural human. Set ELEVENLABS_API_KEY to use ElevenLabs instead, or NEXUS_LOCAL_VOICE=off to skip the neural voice. Speech starts on the first phrase and the next sentences are prepared while one plays.
Gestures. Hand tracking now analyses every camera frame on the GPU with adaptive smoothing. Beyond swipe, pinch, pull/push, circle and two-hand zoom/group/sculpt: swipe up opens the front card and swipe down closes it; point one finger at a card and hold to open it (the cursor ring fills); hold a peace sign to start or stop ambient motion; hold a thumbs up to switch between spatial and human form. Each recognised gesture flashes beside the hand cursor.
Sound effects. Transformations have synthesized sound cues (no audio files). Toggle them with FX in the bottom bar; the choice is remembered.
The core. In spatial mode a gyroscopic singularity sits at the centre of the orbit: a plasma heart inside a scanning event-horizon shell, circled by three tilted rings with light racing along them and particle streams flowing in their planes. The gyroscope spins up while Zeno thinks, everything pulses with the voice while it speaks, and it tilts toward your pointer. Click it to talk to Zeno.
This is the beginning of the README. Read the rest at the original. Read the full README ↗
