Alchemy

A local-first, privacy-respecting desktop app inspired by NotebookLM. Import your sources, chat with them grounded in citations, and generate documents — running 100% on your machine when you want it to. No accounts, no cloud requirement, nothing leaves your laptop by default.

Chat with any model you already have (see docs/RFC-inference-providers.md):

• On this Mac — Apple's on-device model on macOS 26 with Apple Intelligence: fully offline, no setup, retrieval auto-tuned to its window.
• Local servers — Ollama, LM Studio, vLLM, or any OpenAI-compatible localhost endpoint.
• Subscriptions you already pay for — Claude Code, Codex, Gemini CLI, Cursor, OpenCode, GitHub Copilot, Hermes, IBM Bob, Prime Agent, and Pi run headless through their own CLIs; your existing sign-ins carry the bill and answers are captioned with the model (and metered cost when reported).
• API gateways — 30+ presets (OpenAI, Anthropic, NVIDIA, OpenRouter, Groq, …) or any custom OpenAI-compatible URL. See Using a cloud LLM.

Whatever answers, your sources stay on-device: embeddings default to the built-in private embedder, podcast audio is voiced locally by Kokoro, and image OCR runs through your own Ollama vision model.

The decisions remain yours

Alchemy helps hold the thread without taking it over. Sources can be opened, citations lead back to the evidence, and notebooks remain portable as files. Night Shift writes notes and reports. It will not act outward.

Built with Tauri 2 + React front-end, a Rust backend, LanceDB for embedded vector + relational storage, and a Linear-inspired UI with 37 themes.

One minute of Alchemy — full demo with sound: an agent fills a notebook live over MCP, chat cites exact passages, and the notebook becomes a two-host podcast voiced on-device.

---

Features

• Notebooks — a home screen of notebooks (most-recent first); opens to your last one.

• Sources — import PDF, the whole Office family (.doc / .docx / .ppt / .pptx / .xls / .xlsx / .xlsb), OpenDocument (.odt / .ods / .odp), RTF, EPUB, Box Notes (.boxnote), CSV/TSV, images, text, Markdown, paste text, or fetch a URL — including link-shared Google Docs, Sheets, and Slides (paste the link, or drag the .gdoc / .gsheet / .gslides files from a local Google Drive folder). Each is extracted, chunked, and embedded locally. Drag-and-drop onto the Sources panel. File sources remember their on-disk path — Refresh re-reads a changed file (and URL sources re-fetch); Show in Finder jumps to the original. Failed/blocked imports show an error badge and can be retried; edited/refreshed sources are re-embedded.

• Search your Mac — don't remember where a file lives? Type a few characters in Add Source and a Spotlight-backed search returns ranked file and folder hits, addable in one click. Ingestible, name-matching, recently-modified files float to the top, and Trash / node_modules / ~/Library noise is filtered out. Adds route through the same path as a drag-and-drop, so folders become synced folder sources and OKF bundles import exactly as they would otherwise.

• Folder sources — add any folder, or start from a detected cloud sync root (Google Drive, OneDrive, Dropbox, Box, iCloud Drive) with one click and drill down to a subfolder. It scans supported files, keeps syncing on a timer, and lists cloud placeholders without downloading them — iCloud stubs hydrate on their own in the background, and Dropbox Paper (.paper) / Google .gdoc·.gsheet·.gslides stubs resolve to their online docs.

• Mac apps as sources — connect Apple Notes (individual notes), Reminders lists, rolling Calendar windows, and Stocks watchlists via ; they re-sync automatically. Notes can be edited (and reminders added) from inside Alchemy — writes go to the real app and sync back.

Architecture

┌──────────────────────────────── Tauri window ────────────────────────────────┐
│  React + Tailwind                                                            │
│  Home (notebook picker)  |  Sources │ Chat (streaming) │ Studio (docs+notes) │
└───────────────────────────────── IPC (invoke / events) ──────────────────────┘
                                     │
┌───────────────────────────────── Rust backend ───────────────────────────────┐
│  commands.rs   Tauri command surface + per-model stats                       │
│  ingest.rs     extract (anydoc: office/odf/rtf/epub/csv → GFM; pdf/url/text) │
│                → normalize → structure-aware                                 │
│                chunking (paragraphs/headings, title+section context prefix)  │
│  pdf.rs        PDFium page rasterization for scanned-PDF OCR                 │
│  ai/ollama.rs  embeddings, streaming chat, OCR over Ollama HTTP              │
│  agent.rs      agentic "deep research" retrieval loop                        │
│  rag.rs        retrieval prompt assembly + generator prompts                 │
│  router.rs     semantic router: routes questions to likely notebooks         │
│  trace.rs      local retrieval trace JSONL (stages, warnings, citations)     │
│  db.rs         LanceDB tables: notebooks, sources, chunks(+vectors+FTS),     │
│                messages, notes, report_schedules, routes; hybrid search      │
│                (RRF + tie-break), diversity caps, search traces              │
└──────────────────────────────────────────────────────────────────────────────┘
                                     │
                              Ollama (localhost:11434)

The RAG loop: a question is embedded and the chunks table is searched two ways — vector similarity (finds paraphrases) and BM25 full-text (finds exact names, codes, and numbers that embeddings miss) — with the two result lists merged by reciprocal rank fusion (deterministic: score ties break by chunk id, so the same question always cites the same passages). Every retrieval also appends a local trace line (stages, warnings, citations) to traces/retrieval.jsonl, and agents can call the MCP search_debug tool to see exactly why a passage did or didn't surface. Chunks are structure-aware: whole paragraphs packed to ~280 words, headings kept with their sections, and each chunk is embedded with a [Doc title › Section] context prefix (the stored text stays verbatim so citation highlighting still works). The top passages become numbered excerpts in the prompt, and the model answers with [n] citations that map back to the retrieved chunks shown in the UI. See docs/ARCHITECTURE.md.

Deep-research (agent mode) loop

In agent mode (agent.rs), the model plans its own retrieval before answering. Each step it emits one JSON action; evidence accumulates until it decides it has enough:

User question
      │
      ▼
┌─ Planner loop (≤5 steps, one JSON action per step) ──────────────────────────┐
│                                                                              │
│  "search <query>"  hybrid search (vector + BM25, RRF) → top-20 pool          │
│                        │                                                     │
│                        ▼  rerank (one model call)                            │
│                    the ≤5 passages that actually answer ─► evidence pool     │
│                    180-char snippets ─────────────────────► transcript       │
│                                                                              │
│  "read <source>"   fetch full text (shared budget:                           │
│                    120k chars gateway / 12k local)                           │
│                        │                                                     │
│                        ▼  sub-read (one model call)                          │
│                    distill against the question into                         │
│                    verbatim quotes (≤4k chars) ───────► evidence pool        │
│                    same distillate ───────────────────► transcript           │
│                                                                              │
│  "answer"          enough evidence — exit loop                               │
│                                                                              │
│  The transcript (searches + distilled reads so far) is re-sent to the        │
│  planner each step so it can decide what's still missing.                    │
└──────────────────────────────────────────────────────────────────────────────┘
      │  fallback: nothing gathered → direct top-8 vector search
      ▼
┌─ Writer (one streamed call) ─────────────────────────────────────────────────┐
│  grounding rules + persona  │  last 6 chat turns  │  full source manifest    │
│  numbered excerpts [1..n] from the evidence pool                             │
│                └─► streamed answer citing [n], mapped back to sources        │
└──────────────────────────────────────────────────────────────────────────────┘
      │
      ▼
Persisted message — citation snippets are the distilled verbatim quotes, so
the excerpts shown in the UI are exactly what the writer saw, and clicking a
citation highlights real passages in the source.

Because a read returns distilled evidence rather than raw text, the writer's context stays small no matter how many or how large the read documents are — the read budget bounds each sub-read call, not the final prompt.

Studio generators and scheduled reports share the retrieval improvements too: they build their corpus by budgeting fairly across all sources (waterfill), and any source that exceeds its allocation gets the same distillation treatment — the overflow is distilled against the generation instructions and appended, so a big source contributes its relevant passages instead of losing everything past the cut.

Corpus-wide retrieval (Ask Everything)

Meta-chat answers questions across every notebook at once, through a staged pipeline where each stage earned its place in the eval harness (cargo test --lib retrieval_eval -- --nocapture, ranked metrics in docs/RFC-retrieval-maturity.md):

• Semantic routing — one embedded route per source/note title (a notebook ranks as high as its closest item); questions search the top-4 notebooks when the corpus is big enough to benefit. The index self-heals: an unchanged corpus re-embeds nothing.
• Hybrid with an escape hatch — routing narrows only the vector side; BM25 stays corpus-wide, so an exact invoice number or error code the router couldn't see still surfaces. Measured: routed recall equals flat recall with the escape hatch, and drops without it.
• Diversity caps — at most 2 chunks per source, 3 per notebook, 4 notes, with skipped candidates backfilling in score order: one chatty notebook can't crowd out the rest, and shaped search never returns fewer hits than flat.
• Deep search — optionally retrieve a 3× pool and let the chat model keep the passages that actually answer (default on for gateway models, off for local; any failure falls back to fusion order).
• Title fallbacks — "where did I save the contractor agreement?" matches source and note titles directly even when no chunk cracks the top-k.
• Source gists — each source is distilled to a short overview in the background and indexed alongside its chunks; those overview rows join fusion as their own capped class (default 2), so "what's in here?" / "which source covers X?" questions match the right document even when no single passage is a strong hit. A gist hit cites its source and drills down into it; the gates (identifier-overlap, length, degeneracy) keep a bad gist from misrouting.
• Corpus-adaptive k — the retrieval budget grows with corpus size (log curve, +1 per doubling past ~200k chars, capped per model profile), so a 10M-char corpus isn't starved by the k that suits a 50k one. RRF ties break by score, then source recency, then chunk id — determinism preserved.
• Global answers (map-reduce) — broad questions (enumerative/comparative markers, no identifier-shaped token) take a lazy map-reduce path: retrieve gists corpus-wide, run one scoped extract per covered source (bounded fan-out), then synthesize a single answer cited at source granularity with chunk drill-down on demand. Any failure at any step degrades to the pointed path, byte-for-byte.

A deterministic scale fence (eval_scale_fence) makes the corpus-scale claim falsifiable: across synthetic 1M / 3M / 10M-char corpora, exact and paraphrase recall both hold at 1.00 while adaptive k grows 10 → 13 — recall flat from 1M to 10M, a passing eval rather than extrapolation.

Chat tracing flow

The UI traces long-running chat work through Tauri events. ChatPanel starts a send through the Zustand store and api.sendMessage / api.sendMessageAgentic; the backend persists the user turn, emits chat://step progress labels for tooling and agent work, emits chat://token deltas as the model streams, then emits chat://done after the assistant message is saved. The store appends live text into streamingText, records progress labels in steps[], and then reloads canonical messages, sources, notes, and report schedules.

How notebooks sync

Every notebook is a folder of markdown (the Open Knowledge Format), and the folder is the sync. There is no server.

• Where. Settings → Notebooks → Folder. The default is the app's iCloud container, which Finder shows as iCloud Drive › Alchemy (on disk: ~/Library/Mobile Documents/iCloud~com~thrashr888~alchemy/Documents), or ~/Documents/Alchemy when iCloud Drive is off. Point it at a Dropbox or Google Drive folder and that folder is the sync instead.
• What. One folder per notebook: index.md, sources/ with one concept file per source (its extracted text under a single frontmatter block that names the source, its origin, and its sync identity), notes/, references/ for copied originals such as PDFs, sync/ for per-Mac stamps, log.md as a short history, and conflicts/.
• Originals are never written to. A PDF in Downloads, a markdown file in a repo, an Apple Note: Alchemy reads them and writes nothing back. Only the bundle files get Alchemy's frontmatter, and a file has exactly one block — the document's own keys merged with Alchemy's, the newer record winning per key.
• Between Macs. Open the same folder on another Mac and its Alchemy reads the bundle into its own local database; edits made in either app, in the folder by hand, or by an agent flow the other way on the next pass. Rows are matched by sync identity, not path, so renames and moves survive. A change on both sides lands in conflicts/, never over the other copy and kept for 30 days, and nothing is deleted without a deletion record.

Details, decisions, and the recovery rules: docs/RFC-okf-live.md.

Install (Apple Silicon)

With Homebrew:

brew tap thrashr888/tap
brew install --cask thrashr888/tap/alchemy

Or download the latest Alchemy_x.y.z_aarch64.dmg from Releases, open it, and drag Alchemy to Applications. Builds are signed with a Developer ID and notarized by Apple, so they open with a normal double-click.

Requires Ollama running locally.

Prerequisites (development)

• Ollama running locally (ollama serve).

• Models pulled — for example:

  ollama pull mxbai-embed-large       # embeddings
  ollama pull gpt-oss:20b             # chat (or any chat model)
  ollama pull glm-ocr                 # OCR (optional, for images / scanned PDFs)
  

• Rust (stable) and Node with pnpm. protoc is required to build LanceDB (brew install protobuf).

• The PDFium library (for scanned-PDF OCR) is fetched automatically on first build by scripts/fetch-pdfium.sh (via build.rs); no manual step needed.

Develop

pnpm install
pnpm tauri dev

The first build compiles LanceDB and is slow; subsequent builds are fast.

Test & lint

cd src-tauri
cargo test          # unit tests (+ a graceful-skip Ollama integration test)
cargo fmt -- --check
cargo clippy --all-targets -- -D warnings

CI (.github/workflows/ci.yml) runs the frontend typecheck/build plus the above on every push and PR.

Configuration

Open Settings (gear icon) to set the Ollama URL and choose models. Defaults:

Setting	Default
Ollama URL	http://localhost:11434
Chat model	gpt-oss:20b
Embedding model	mxbai-embed-large
Vision model	(unset — OCR disabled)

Switching the embedding model prompts to re-embed all sources (models produce incompatible vectors), so retrieval never silently breaks. Data is stored in the OS app-data directory under lancedb/.

Using a cloud LLM (any OpenAI-compatible gateway)

Ollama stays the default, but Alchemy can route chat, generation, deep research, and OCR through any OpenAI-compatible gateway instead — useful on machines that can't run local models (e.g. a 16 GB laptop). The same path works for LM Studio, vLLM, LiteLLM, OpenRouter, enterprise gateways, or Ollama's own /v1 endpoint.

With a gateway selected, sources still stay local: embeddings run on the built-in CPU embedder (a ~30 MB model downloaded on first use, no Ollama required), so nothing but your chat turns leaves the machine.

Configuring it in Alchemy

During onboarding (or later in Settings → Models):

1. Switch the provider to OpenAI-compatible gateway.
2. Enter the gateway's base URL (usually ends in /v1) and paste your API key.
3. Click Save & check — Alchemy lists the gateway's models, auto-selects one, and shows Connected. Pick a different model from the dropdown any time.

Keys are stored only in the local ai_config.json and sent solely as an auth header to the gateway you configure — never anywhere else. LiteLLM-style gateways with non-standard key schemes are handled automatically. Alchemy does no in-app accounting — usage is whatever your gateway bills.

Provider quick-reference

Any OpenAI-compatible endpoint works. Settings that are known-good:

Provider	Gateway URL	API key	Notes
OpenAI	https://api.openai.com/v1	sk-… from platform.openai.com	gpt-4o & friends are vision-capable (works for OCR too)
Anthropic	https://api.anthropic.com/v1	sk-ant-… from the console	Via Anthropic's OpenAI SDK-compat layer; sk-ant- keys get the right headers automatically
OpenRouter	https://openrouter.ai/api/v1	sk-or-… from openrouter.ai/keys	One key, hundreds of models (anthropic/…, google/…, …)
Groq	https://api.groq.com/openai/v1	from console.groq.com	Very fast open-weight models
IBM Bob	https://api.us-east.bob.ibm.com/inference/v1	bob_… from the Bob portal (IBM-internal)	Bob's Apikey scheme & team headers are handled automatically
LM Studio	http://localhost:1234/v1	(none)	Local server; load a model in LM Studio first
Ollama	http://localhost:11434/v1	(none)	Ollama's own OpenAI-compat endpoint (or just use the native Ollama provider)

You can leave the Gateway URL empty for OpenAI, Anthropic, OpenRouter, Groq, and Bob-style keys — Alchemy infers the URL from the key format.

For OCR of images and scanned PDFs, set a vision-capable model as the gateway vision model (e.g. gpt-4o, a Claude model, or any vision model on OpenRouter); leaving it empty disables OCR. Embeddings never leave your machine regardless of provider — the built-in on-device embedder (potion-base-8M) indexes your sources.

Releases

Releases are cut locally on Apple Silicon — one command builds, signs, notarizes, and publishes:

scripts/release.sh 0.4.2

See RELEASE.md for what it does, one-time setup (Developer ID cert + notary profile), and the manual CI fallback (.github/workflows/release.yml, triggered from the Actions tab). Builds are signed with a Developer ID and notarized, so the .dmg opens with a normal double-click.

The app bundles the PDFium library (for scanned-PDF OCR). An Intel (x86_64) build is possible on a macos-13 runner but those runners queue for hours; it can be re-added to the matrix if needed.

Scope

Video overviews are intentionally out of scope. Notes are not embedded into retrieval on their own — Convert to source to make a note retrievable.