Corella
A self-hosted meeting assistant: it records a call from your browser (or takes an upload), transcribes and speaker-labels it, keeps a live AI copilot grounded in your own documents, and turns the call into a searchable transcript plus a post-call summary and coaching report. Runs entirely on your own Linux box via Docker Compose — your own database, your own vector store, and (optionally) your own local LLM, with no required cloud dependency beyond whichever hosted LLM/STT provider you choose to connect.
Features
- Live in-browser recording — captures your mic and, optionally, a shared browser tab's audio, transcribing both sides as the call happens.
- Upload-based transcription — drop in an existing recording (most common audio formats, incl.
.caf) for the same pipeline, offline.
- Speaker separation, live — more than one voice on your own mic (an in-person meeting around one laptop) or on the shared tab audio gets split into "Speaker 1"/"Speaker 2" / "Them 1"/"Them 2" mid-call, not just after the fact.
- Cross-meeting voice recognition — enroll your voice once and Corella recognizes you (and teammates in the same organization) across future calls; unrecognized speakers get identified live from what they say ("Hi, this is Lucas") via your configured LLM.
- Pluggable speech-to-text — local
faster-whisper by default (zero config), or Deepgram if you connect an API key — per-user, per-provider model overrides available in Settings.
- Pluggable copilot LLM — Anthropic, OpenAI, Gemini (bring your own key), or a self-hosted Ollama instance — live suggestions, blockers, action items, a live coach score, and a closed-set sentiment during the call.
- Post-call reports — auto-generated the moment a call finishes: title, summary, key topics, sentiment, notable quotes, action items, talk share, and a coach score, tuned by call type (sales/support/interview/1:1/meeting).
- Knowledge base — upload your own documents; the live copilot retrieves relevant snippets via semantic search.
- Semantic search — across your own meeting history, across your group's shared reports, or (as an org owner/admin) everything in the active organization.
- Organizations — the isolation boundary. Open signup creates
{Name} Org with you as owner; closed mode uses one instance org. Groups nest inside an org. Owner/admin/member are org roles; super_admin is an instance flag.
- Groups — org-scoped subcategories. A shared knowledge base (org owner/admin upload and maintain it, assigned per group) and report-only (not raw transcript) visibility into a group-mate's calls. Voice recognition is org-wide so multi-group users aren't stuck.
- Organization settings — members, invites, groups, secrets, call types, and org spend. Super admin is a separate instance console (org list, super-admin flags, instance-wide costs).
- Org-admin live debug panel — while recording your own call as an org owner/admin or super admin, toggle a technical event stream (VAD flushes, STT/LLM request+response timing, diarization dispatch) for in-the-moment debugging.
- Per-call cost estimate — a best-effort running total per meeting, from real token usage (LLM calls) and real audio duration (Deepgram STT) where the provider reports it, with a by-provider breakdown on the organization cost dashboard.
- API access — API keys (Settings) for creating/reading meetings and streaming a live recording from another system, plus per-call-type hooks that fire before a call starts (pull in external context) and after it finishes (push the full result out) — see
API.md. A website can use packages/corella-live instead of speaking the WebSocket protocol by hand.
Architecture
flowchart TB
Browser["Browser\n(mic + tab audio)"] --> Web["web\nReact SPA (nginx)"]
Web -->|"REST + WebSocket"| API["api\nFastAPI"]
API --> Postgres[("postgres")]
API --> Redis[("redis\nqueue + pub/sub")]
Redis --> Worker["worker\nCelery: full diarize()\npipeline, embeddings"]
Worker --> Qdrant[("qdrant\nvector search\n(knowledge base +\nmeeting search)")]
- api — FastAPI. Auth, meeting/KB/org/admin CRUD, WebSocket audio ingestion and live event push (transcript, copilot, diarization updates, admin debug events). Runs
faster-whisper directly for live transcription (torch-free, light enough for this process), and also loads pyannote.audio's speaker-embedding model directly (not gated, unlike the full diarization pipeline below) for the instant "does this match an already-confirmed voice" check on the hot live-recognition path — no worker round-trip for that. When a session's STT is Deepgram, its own native diarization (word-level speaker index) drives live same-room/same-tab speaker splitting entirely in this process too, with zero worker involvement while that stream stays healthy. Everything else torch-dependent (the full diarize() pipeline, offline transcription's diarize step) stays worker-only.
- worker — Celery. Runs the heavier/blocking jobs: offline transcription + diarization for uploads, the periodic full-
diarize()-pipeline reconciliation pass live sessions use when their STT isn't Deepgram (or as a fallback if a Deepgram stream drops), voice-identity matching, knowledge-base/meeting-search embedding, report generation, and voice enrollment — so none of it blocks the API process or the live WebSocket loop.
- postgres — structured data: users, organizations, memberships, invites, groups, meetings, transcript segments, speakers, voice identities, action items, provider/STT credentials, per-call LLM usage ledger.
- qdrant — vector search, three collections: knowledge-base document chunks, meeting-transcript chunks (search), and speaker voice embeddings (cross-meeting recognition).
- redis — Celery broker/result backend, plus pub/sub for bridging worker-side events (diarization, live labels) back to the right live WebSocket connection.
- web — React/TypeScript SPA, built static and served by nginx.
The copilot LLM and the speech-to-text engine are each pluggable per user: a per-user saved key takes priority, then an instance-wide .env fallback, then — for STT only — local faster-whisper, which needs no key at all. See BRANDING.md for the UI's visual language if you're touching web/, or docs/AUDIO_PIPELINE.md for a detailed look at the transcription/live-streaming/speaker-diarization pipeline — including the full decision flow and the real bugs that shaped it — if you're touching app/services/vad/, app/services/diarization/, app/services/asr/, app/ws/live_session.py, or app/workers/tasks.py.
Tech stack
| |
|---|
| Backend | Python, FastAPI, SQLAlchemy 2 (async) + Alembic, Celery |
| Speech/ML | faster-whisper, Deepgram (optional), pyannote.audio (diarization + speaker embeddings), sentence-transformers / fastembed (KB + meeting search embeddings) |
| LLM clients | Hand-rolled httpx clients for Anthropic, OpenAI, Gemini, Ollama — no SDK dependency |
| Data | PostgreSQL, Qdrant (vectors), Redis (broker + pub/sub) |
| Frontend | React, TypeScript, Vite, Tailwind CSS, react-router |
| Infra | Docker Compose (CPU-first; optional NVIDIA override) |
Repository layout
corella/
server/
app/
api/ REST routers — auth, meetings, kb, settings, organizations, admin
ws/ WebSocket live-session handler
core/ config, security (JWT + secret encryption), db sessions
models/ SQLAlchemy models
schemas/ Pydantic request/response schemas
services/
asr/ faster-whisper + Deepgram clients, provider resolution
diarization/ pyannote.audio wrapper, online speaker clustering
alignment/ merges ASR output + diarization into labeled utterances
llm/ provider clients (anthropic/openai/gemini/ollama) + resolution
embeddings/ sentence-transformers/fastembed + Qdrant collections
copilot/ live suggestions, post-call report generation
admin/ cost-analytics aggregation
audio/ WAV I/O, channel mixing/windowing
vad/ speech/silence utterance detection
workers/ Celery task definitions
main.py
alembic/ DB migrations
web/
src/
routes/ one file per screen (Dashboard, Organization, Invite, Super admin, …)
components/ shared UI (AppShell, …)
lib/ typed API client, live-session WS client, auth context
public/
docker-compose.yml CPU-first stack
docker-compose.gpu.yml optional NVIDIA runtime override
.env.example
BRANDING.md UI visual language + component conventions
VERSIONING.md three-digit version guide — when to bump what
docs/
AUDIO_PIPELINE.md transcription/live-streaming/diarization deep dive
packages/
corella-live/ npm client for the live recording API
Running it
cp .env.example .env # fill in JWT_SECRET at minimum
docker compose up --build
If you have an NVIDIA GPU + the NVIDIA Container Toolkit installed, layer on the GPU override for faster/larger transcription and diarization models:
docker compose -f docker-compose.yml -f docker-compose.gpu.yml up --build
Deploying an update
docker compose build api worker web
docker compose up -d api worker web
Migrations run automatically on api startup. postgres/redis/qdrant don't need rebuilding for an app-code change.
Configuration
See .env.example for the full, documented list. Highlights:
- Core:
JWT_SECRET (required), CORS_ORIGINS, ENVIRONMENT, API_PORT / WEB_PORT (host-side published ports in docker-compose.yml, default 8090 / 8080), PUBLIC_APP_URL (identifies this instance in pre/post call-type API hooks — see API access; must match the web origin). Changing API_PORT without setting PUBLIC_API_URL is enough — the web image defaults to http://localhost:${API_PORT}. Changing WEB_PORT means updating CORS_ORIGINS and PUBLIC_APP_URL to the new origin.
- Access control:
ALLOW_PUBLIC_REGISTRATION, MAX_ORGS_PER_USER, INVITE_EXPIRE_DAYS, optional RESEND_API_KEY / RESEND_FROM_EMAIL (email the invite link; copy-link still works without them), ADMIN_EMAIL/ADMIN_PASSWORD (bootstrap super_admin, see Access control below).
- Data stores:
DATABASE_URL, REDIS_URL, QDRANT_URL — defaults match docker-compose.yml's service names, only change these if you're pointing at externally-hosted stores.
- Speech:
HF_TOKEN (diarization, see below), WHISPER_MODEL/WHISPER_COMPUTE_TYPE, optional DEEPGRAM_API_KEY/DEFAULT_MODEL_DEEPGRAM.
- Storage:
AUDIO_STORAGE_PATH/MAX_AUDIO_UPLOAD_MB, KB_STORAGE_PATH/MAX_KB_UPLOAD_MB, EMBEDDING_MODEL.
- LLM providers (all optional instance-wide fallbacks — every provider is also configurable per-user in Settings):
ANTHROPIC_API_KEY, OPENAI_API_KEY, GEMINI_API_KEY, OLLAMA_BASE_URL.
Every credential (LLM and STT alike) follows the same precedence: the signed-in user's own saved key first, then the instance-wide .env value, and — for speech-to-text specifically — local faster-whisper as a final, always-available fallback that needs no key at all. A user can also pin an explicit provider/model/language override in Settings instead of relying on that automatic priority order.
Speaker diarization setup
Speaker diarization uses a gated pyannote.audio pipeline. A valid HF_TOKEN alone isn't enough — the Hugging Face account behind it must also individually accept the terms on each gated model page the pipeline depends on internally (a separate, one-time click-through per repo):
Skipping this doesn't break anything — transcription still works fine and the meeting still finishes, just without speaker labels for uploaded/offline recordings; the pipeline just fails per-file with a clear error in the worker logs instead of loading. Live, same-room/same-tab speaker splitting during a local-whisper (or Deepgram-dropped-mid-session) recording depends on this same gated pipeline for its periodic full diarization pass, so it's subject to the same requirement — a session without HF_TOKEN just skips live labeling there and keeps showing "Me"/"Them". (The instant per-segment recognition check uses pyannote/wespeaker-voxceleb-resnet34-LM, a separate, non-gated model — it works with no HF_TOKEN at all, it just has nothing to recognize against until the gated pipeline has labeled at least one speaker.)
Deepgram STT sidesteps all of the above for live diarization. When a session's speech-to-text is Deepgram (a connected key, per-user or instance-wide), same-room/same-tab speaker splitting uses Deepgram's own native diarization instead — no HF_TOKEN, no gated models, no worker involvement at all for that path. The tradeoff: Deepgram's speaker index is local to that one connection, so it doesn't feed Corella's cross-meeting/group voice-identity recognition (enrollment/name-spotting) the way the pyannote-embedding path does — a session on Deepgram gets instant "Speaker 1"/"Speaker 2" labels, but not recognized-by-name identities, this round.
Recording a meeting
Two ways to get a meeting transcribed, both going through the same pipeline downstream:
- Upload: on the Dashboard, "Upload recording" picks an audio file (most common formats, incl.
.caf), which is transcribed — and, if HF_TOKEN's terms are accepted, speaker-diarized — in the background. The meeting page polls and updates itself as processing finishes, and a summary report generates automatically once it's ready.
- Live recording: "Record live" captures your mic (and, optionally, a shared browser tab's audio) directly in the browser, transcribing each side as it happens, with live AI copilot suggestions (if an LLM provider is connected) and live speaker separation on both channels. Stopping finalizes the recording the same way an upload does — including the automatic report.
Speech-to-text uses whichever engine is currently resolved for the recording user (Deepgram if configured, else local) — see Configuration above for the precedence, or Settings' "AI models in use" panel to see and change what's active.
Access control
Two deployment modes (ALLOW_PUBLIC_REGISTRATION, plus MAX_ORGS_PER_USER, default 1):
- Open (
true): signup creates {full_name} Org and the user is its owner. They may create more orgs up to MAX_ORGS_PER_USER (counts owned orgs, not memberships) and invite others.
- Closed (
false): no self-serve signup and no self-serve org create. Set ADMIN_EMAIL / ADMIN_PASSWORD — that account is created on first startup as super_admin and owner of one instance org (is_instance_org). Admin-created or invited users join that org as members and do not get a personal org.
Flipping the flag later does not merge or split orgs; it only changes whether new signups and org-creates are allowed. ADMIN_EMAIL/ADMIN_PASSWORD only ever create the bootstrap account — changing them later and restarting won't touch an existing password.
Roles are two enums, never one:
User.is_super_admin — instance operator. Bootstrap account, and anyone a super admin promotes. Powers: Super admin console, cross-org support read (GET /api/meetings/all and /search/all), promote/demote super admins. Not implied by creating an org.OrganizationMembership.role — owner | admin | member.
- Owner (exactly one): invite/remove (cannot be removed/demoted by others), groups, KB, call types, secrets, org costs, promote/demote admin↔member, transfer ownership, delete org (never the instance/default org, and never the last remaining org).
- Admin: same day-to-day as owner except cannot demote/remove/replace the owner, cannot transfer, cannot delete the org.
- Member: own meetings plus group-shared surfaces (report-only for groupmates).
The web app picks the active org (PUT /api/organizations/current); every browser request runs in that org. API keys do not switch — each key is bound to one org at creation.
Invites are copy-link tokens (INVITE_EXPIRE_DAYS, default 7). If RESEND_API_KEY and RESEND_FROM_EMAIL are set, creating or resending an invite also emails that link; a Resend failure is logged and the invite still works. Accepting works even when public registration is closed. A new email creates an account and joins that org (no auto-created personal org); an existing account adds a membership.
Writes (delete meeting, report, action-item edits) stay owner-only. Org owner/admin get today's instance-admin powers inside the active org (org-wide meeting list, full read, KB upload/delete, call types, secrets, org cost dashboard).
API access
Corella can be integrated with an external system in three ways — full reference (auth, REST/WebSocket shapes, hook payloads, examples) in API.md. For a website that should stream a live conversation, use packages/corella-live (npm install corella-live) so you do not reimplement the WebSocket quirks (speaker labels in particular).
The three types:
- API keys (Settings → API keys) — a long-lived credential that acts as its owner in the organization it was created in. Switching orgs in the UI does not move the key. Each key has a max live-session duration (default 60 minutes) so a hung integration can't record forever.
- Live streaming — the same WebSocket protocol the browser app uses to record is reachable by API key too; no separate streaming endpoint exists.
- Pre/post call-type hooks (Organization → Call types) — an org owner/admin can configure an external API call to fire before a call of a given type starts (optionally feeding the response back into the live copilot's context, alongside the knowledge base) and/or after it finishes (optionally sending the full transcript/report/coaching timeline out to another system). Every such request carries four fixed headers (
X-Corella-App-Url, X-Corella-Meeting-Id, X-Corella-User-Id, X-Corella-Org-Id) that can't be overridden by custom header config.
Design & branding
The UI follows a deliberate, documented visual language — see BRANDING.md for the full color/type/component reference before making frontend changes. In short: near-white/charcoal surfaces, a single navy accent, a serif for headlines only, border-first cards with minimal shadow.
Development
Contributing? See CONTRIBUTING.md for setup, this project's conventions (credential handling, migrations, adding a new LLM/STT provider), and what a good PR looks like here — and VERSIONING.md for which digit to bump on a main → release cut.
Backend:
cd server
pip install -e ".[worker]"
alembic upgrade head
uvicorn app.main:app --reload
Frontend:
cd web
npm install
npm run dev
Type-check and build the frontend before committing:
cd web
npx tsc -b && npx vite build
A backend pytest suite (server/tests/) covers the highest-value logic — permission boundaries, credential/provider resolution, webhook templating, pricing math — not every endpoint; growing it is a welcome contribution. ruff and pytest are both required checks on any PR into main/release, and ruff alone runs on every push for fast feedback (.github/workflows/). Frontend correctness is still tsc -b + vite build, no component-test suite yet. Beyond what's covered by these, changes are verified by hand against a real, isolated Docker stack (see CONTRIBUTING.md and any recent commit message for the pattern) — several real bugs in this project's history were only ever caught that way, not by a unit test.
Status
Actively developed. Done so far: organizations with owner/admin/member plus instance super_admin, copy-link invites, auth (open signup creates {Name} Org; closed mode uses one instance org), upload and live in-browser recording with speaker-labeled transcripts, pluggable LLM copilot (live suggestions/blockers/action items/coach score) and pluggable STT (local or Deepgram), knowledge-base ingestion and semantic meeting search, auto-generated post-call reports (summary/topics/sentiment/quotes/coach score) tuned by org-scoped call type, org-scoped voice recognition with live LLM name-spotting, live same-room and same-tab speaker separation, organization settings (people/groups/invites/secrets/call types/costs) plus a super-admin console, and per-call cost tracking against a real usage ledger.
Not yet built: post-call "polish" re-transcription with a larger model.
Security
To report a vulnerability, see SECURITY.md. Do not open a public issue.
License
Apache License 2.0.
