JARVIS

Just A Rather Very Intelligent System.

A voice-first AI assistant that runs on your Mac. Talk to it, and it talks back -- with a British accent, dry wit, and an audio-reactive particle orb straight out of the MCU.

JARVIS connects to your Apple Calendar, Mail, and Notes. It can browse the web, spawn Claude Code sessions to build entire projects, and plan your day -- all through natural voice conversation.

"Will do, sir."

This fork by Navii extends Ethan Rogers' original JARVIS to keep working fully offline and to run your Mac. Original work © Ethan Rogers, used under its non-commercial license.

What's New in This Fork

🧠 Offline brain	Falls back to local Ollama models when the Claude API is unreachable or out of credits (ollama_fallback.py)
👂 Always-on wake word	Background daemon: mic → voice-activity detection → local faster-whisper → "Hey JARVIS" (jarvis_daemon.py)
🗣️ Offline British voice	Piper en_GB-alan neural TTS between edge-tts and macOS say, so it never goes silent (local_tts.py)
🎙️ Local speech-to-text	Whisper on-device, no Google servers (stt.py)
🖥️ Mac control	Volume, brightness, apps, battery, Wi-Fi, lock, iMessage (system_control.py)
📇 More Apple apps	Contacts, Reminders, Spotify, clipboard, plus GitHub and Home Assistant
⏰ Triggers	"Alert me when NVDA drops below 500", "remind me at 3pm" (trigger_monitor.py)
📈 Briefings & portfolio	Voice-ready morning brief and holdings P&L (morning_briefing.py, portfolio.py)
🔁 Learns your routine	Time-of-day patterns pre-warm answers; weekly self-review (routine_learner.py, jarvis_self_improve.py)
🛎️ Works with the browser closed	macOS notifications + spoken fallback (notification_bridge.py)

---

What It Does

• Voice conversation -- speak naturally, get spoken responses with a JARVIS voice
• Builds software -- say "build me a landing page" and watch Claude Code do the work
• Reads your calendar -- "What's on my schedule today?"
• Reads your email -- "Any unread messages?" (read-only, by design)
• Browses the web -- "Search for the best restaurants in Austin"
• Manages tasks -- "Remind me to call the client tomorrow"
• Takes notes -- "Save that as a note"
• Remembers things -- "I prefer React over Vue" (it remembers next time)
• Plans your day -- combines calendar, tasks, and priorities into a plan
• Sees your screen -- knows what apps are open for context-aware responses
• Audio-reactive orb -- a Three.js particle visualization that pulses with JARVIS's voice

Requirements

• macOS (uses AppleScript for Calendar, Mail, Notes integration)
• Python 3.11+
• Node.js 18+
• Google Chrome (required for Web Speech API)
• Anthropic API key -- powers the AI brain (get one here)
• Fish Audio API key -- powers the voice (get one here)
• Claude Code CLI -- for spawning dev tasks (install here)

Quick Start (with Claude Code)

The fastest way to get running:

git clone https://github.com/yourusername/jarvis.git
cd jarvis
claude

Claude Code will read the project's CLAUDE.md and walk you through setup step by step -- API keys, dependencies, SSL certs, everything.

Manual Setup

# 1. Clone the repo
git clone https://github.com/yourusername/jarvis.git
cd jarvis

# 2. Set up environment
cp .env.example .env
# Edit .env with your API keys (see below)

# 3. Install Python dependencies (PyAudio needs PortAudio first)
brew install portaudio
pip install -r requirements.txt

# 4. Install frontend dependencies
cd frontend && npm install && cd ..

# 5. Generate SSL certificates (needed for secure WebSocket)
openssl req -x509 -newkey rsa:2048 -keyout key.pem -out cert.pem -days 365 -nodes -subj '/CN=localhost'

# 6. Start the backend (Terminal 1)
python server.py

# 7. Start the frontend (Terminal 2)
cd frontend && npm run dev

# 8. Open Chrome
open http://localhost:5173

Click the page once to enable audio, then speak. JARVIS will respond.

Configuration

Edit your .env file:

# Required
ANTHROPIC_API_KEY=your-anthropic-api-key-here
FISH_API_KEY=your-fish-audio-api-key-here

# Optional -- your name (JARVIS will address you personally)
USER_NAME=Tony

# Optional -- specific calendar accounts (comma-separated)
# Leave empty to auto-discover all calendars
[email protected],[email protected]

Architecture

Microphone -> Web Speech API -> WebSocket -> FastAPI -> Claude (Haiku) -> Fish Audio TTS -> WebSocket -> Speaker
                                                |
                                                v
                                        Claude Code Tasks
                                        (spawns real dev work)
                                                |
                                                v
                                        AppleScript Bridge
                                        (Calendar, Mail, Notes, Terminal)

Layer	Technology
Backend	FastAPI + Python (server.py, ~2300 lines)
Frontend	Vite + TypeScript + Three.js
Communication	WebSocket (JSON messages + binary audio)
AI (fast)	Claude Haiku -- low-latency voice responses
AI (deep)	Claude Opus -- research and complex tasks
TTS	Fish Audio with JARVIS voice model
System	AppleScript for all macOS integrations

This is the beginning of the README. Read the rest at the original. Read the full README ↗