Mumble 🦥
Mumble at your phone all day. Open models sort it out overnight.
Live: https://mumble-azure.vercel.app
Record rambling voice memos whenever something pops into your head. At 3 AM, a
free GitHub Actions runner wakes up, transcribes them with Whisper, and has
Gemma turn them into to-dos, calendar events and notes. In the morning you open
one screen and tap Done, Keep, Add to calendar or Dismiss (each with Undo).
Until then you can play a memo back to check it saved; kept notes and search live in Remember.
It costs nothing to run.
A real morning
A demo week of everyday memos (voiced with macOS say): 9 memos, 70 seconds of
talking, 220 words. The nap turned them into 16 to-dos, events and notes, and
Morning opened with:
10 min saved. Last night: 9 memos, 1 min of mumbling instead of 11 min of typing and sorting.
Mumble counts typing at 40 words a minute, plus about 20 seconds to file each item
into the right list with the right date.
On Sundays, Ultramarine reads the whole week. In the demo week it noticed "call the
dentist" said on three separate mornings, and never done. Every claim cites the
items it's based on. That review cost $0.0004.
How it works
- Open models do all the thinking: faster-whisper for speech,
Gemma (
gemma4:e2b-it-qat) for structure and embeddinggemma for memory, all served by Ollama.
- Gemma never does date math. It copies the exact time phrase you said ("Tuesday at 10am");
plain code turns that into a timestamp. A phrase you never said is thrown away.
- Slow on purpose. CPU only. Nobody is awake at 3 AM to wait for it.
- Your audio doesn't stick around. Each recording is deleted as soon as it's sorted.
- Real sign-in, and everyone only sees their own. Auth0 (OAuth 2 with PKCE) issues
one-hour access tokens; the API checks signature, issuer, audience and app on every
request, and every query is scoped to the signed-in account.
- Quotas, so a free stack stays free. Each account gets 20 memos and 6 MB of audio a
day, 40 unsorted memos, 120 requests a minute, 25 memos per night's nap and one review a
week. Sign-ups stop at 50 accounts. Delete your account and everything goes with it.
- One thing leaves for a closed model, and only if you turn it on. The weekly review sends
that week's item titles and details (never audio or transcripts) to Ultramarine by
Paper Instruments. No
PAPERINSTRUMENTS_API_KEY, no review, and nothing leaves.
Phone PWA ──▶ Vercel (free static site) ◀──▶ Auth0 (sign-in, free)
│
└─ /api ──▶ Vercel Function (Python, free) ──▶ Postgres + pgvector (Tiger Data, free)
▲
3 AM cron ──▶ GitHub Actions (free CPU runner): Ollama + Gemma + Whisper
| Piece | Where | Cost |
|---|
Phone app (web/) | Vercel static files | free (Hobby) |
API (api/index.py → mumble/api.py) | Vercel Function | free (Hobby) |
| 3 AM alarm | GitHub Actions schedule | free |
Night shift (mumble/nap.py) | GitHub Actions, public repo | free |
| Database | Tiger Data (Postgres + pgvector) | free tier |
| Sign-in | Auth0 | free plan |
Project layout
web/ the phone app (PWA): plain HTML, CSS and JS, no build step
api/index.py Vercel's entrypoint; hands each /api request to mumble/api.py
mumble/
core.py the API's logic and quotas, shared by the cloud and the local server
api.py the cloud API (only needs psycopg and PyJWT)
auth.py checks Auth0 access tokens
main.py the local dev server (FastAPI)
nap.py the night shift: Whisper, then Gemma, then embeddings
transcribe.py / structure.py speech to text / text to to-dos, events, notes
review.py the Sunday review by Ultramarine
db.py, config.py
scripts/ demo_week.py seeds a week of spoken memos; claim.py adopts pre-sign-in data
tests/
.github/workflows/nap.yml the 3 AM schedule
vercel.json hosting: static web/, /api rewrite, security headers
Run it locally
Needs uv, Ollama, ffmpeg and Docker.
ollama pull gemma4:e2b-it-qat && ollama pull embeddinggemma
docker compose up -d # local Postgres + pgvector
cp .env.example .env
uv sync --all-groups # the API alone needs psycopg + PyJWT; "nap" has the models
MUMBLE_DEV=1 uv run uvicorn mumble.main:app --reload
Open http://localhost:8000, record something, then tap Can't wait? Nap now
(or run uv run python -m mumble.nap). MUMBLE_DEV=1 signs everything from this
machine in as one local account; set the AUTH0_* variables instead to sign in for real.
The microphone only works on localhost or over HTTPS. Locally, search also
matches by meaning (the query is embedded with Ollama); the serverless API
searches by keyword.
Deploy (all free)
- Database. Create a free Tiger Cloud service and copy its connection string.
Create the tables once:
DATABASE_URL='postgres://...' uv run python -c "from mumble import db; db.init()"
- Sign-in. Create a free Auth0 tenant with a Single Page Application (callback, logout
and web-origin URLs = your app's URL, refresh token rotation on) and an API (identifier
https://<your-app>/api, RS256, token lifetime 3600 s, offline access on). Put the domain,
client id and identifier in web/config.js.
- Night shift. Push this repo to GitHub (public, so Actions minutes are free) and add
the secret
DATABASE_URL under Settings → Secrets and variables → Actions.
- App + API. Import the repo in Vercel (Add New → Project, Hobby plan, no card).
vercel.json already says what to do: serve web/, run api/index.py for /api/*.
Add DATABASE_URL, AUTH0_DOMAIN, AUTH0_AUDIENCE, AUTH0_CLIENT_ID and
MUMBLE_ICS_SECRET (32+ random characters, signs calendar links), then deploy.
Optional: MUMBLE_MAX_ACCOUNTS (default 50), MUMBLE_ALLOWED_EMAILS (invite-only).
- On your phone, open the app, sign in, and Add to Home Screen.
To give one account more room: UPDATE accounts SET quota = '{"memos_per_day": 50}' WHERE email = '...'.
To switch one off: UPDATE accounts SET disabled = true WHERE email = '...'.
Tests
uv run pytest
License
MIT
