mailrag

Ask questions of your own email, on your own hardware, on open models, with nothing required to leave your network.

Why this exists

The first time I pointed cloud AI at my inbox it felt like a superpower — until I thought about what it actually required: handing my entire email history to someone else's servers to make it searchable. For real correspondence — contracts, receipts, the record of who agreed to what — that's a non-starter.

So I built the opposite. mailrag runs on your own hardware, on open models, with nothing required to leave your network. No mailbox upload, no vendor to trust with the whole archive.

Interestingly enough, the lack of public mailbox data seems to back that up. Nobody has ever published their own private mailbox, so every email corpus available for training or evaluation exists because someone lost control of one. That is why a dataset from 2001 is still the field standard twenty-five years later. docs/WHY_LOCAL.md sets it out in full, including what running locally costs you.

The idea: a conversation is the unit of truth

Generic RAG treats every email as an isolated document, and that is the mistake. Most single messages cannot be answered alone. "Sounds good, go ahead" means nothing without the three messages above it. So the unit of truth for a mailbox is the thread, and the rest follows from that:

1. Match small, answer big. Retrieve one message, then answer from its entire reconstructed conversation. Biggest win in the system, and it needs no LLM at all.
2. Hybrid retrieval. bge-m3 dense plus learned-sparse, fused with RRF. You get the concept and the rare exact token that dense vectors lose, like an invoice number or a ticket ID.
3. Local by default. Cloud is a swappable option at two seams, the LLM and the embedder, rather than a requirement.

"Better or worse than ours?" is a real Enron email, and it answers a real question. Embedded on its own it is a bag of five common words that no query will ever surface. Neither will its thread, until you already know what "ours" refers to. So each email gets embedded together with a summary of what came before it, which is what makes the terse reply findable at all. Finding the conversation is the hard half, and roughly 40% of the questions in this project's evaluation depend on a message that terse. docs/RETRIEVAL_GUIDE.md has the long version.

Try it

Nothing here asks you to commit up front. Each step costs a little more than the last, and you can stop at any of them:

	what you need	what you get	your mail leaves?	cost
1. Read	nothing	the measured numbers, below		none
2. just bench	Docker, a few GB of model weights	re-run those numbers yourself		none
3. just demo	same	both levers measured: findability and completeness		none
4. Your mail + your agent	IMAP or .eml, an MCP client	your archive, answerable by Claude/ChatGPT	yes, to that provider	your LLM's usage
5. Your mail + local model	~9 GB RAM/VRAM	the same, fully airgapped	no	electricity

Steps 2 and 3 need no API key and no private data. Most evaluations should stop at 3. That is enough to judge whether the retrieval is any good.

git clone https://github.com/fmasi/mailrag.git
cd mailrag
conda create -y -n mailrag python=3.12 && conda activate mailrag
just install                           # poetry install --with dev --all-extras --no-root
just demo                              # two indexes, same questions: see what context buys
just bench                             # the full retrieval benchmark

Prerequisites are conda with Python 3.11 to 3.13, Poetry 2.x, Docker (for the Qdrant container) and just (brew install just). The first run pulls a few GB of bge-m3 weights. Neither command needs an API key, an LLM endpoint, or a .env file, and both run on data committed in this repo.

just demo builds two indexes over the same 1,200 public Enron emails, one plain and one with each message embedded alongside its conversation's preceding context, then asks 99 single-message questions and 73 spanning ones:

Q: "who was left off the first distribution list?"
The message that answers it:  "Sandi: Apologies. Inadvertently didn't
                               include you on first..."
  plain index    -> not in top 20
  with context   -> rank 1

index	R@1	R@5	R@10
plain	37.4%	60.6%	74.7%
with thread context	50.5%	73.7%	80.8%

Both arms answer identical queries, so the paired test is the one that counts: context fixes 16 questions and breaks 3 at R@5, McNemar exact p = 0.0044. That is findability.

The second half is completeness, and it needs a different question. On the 73 spanning questions the right conversation is found 97.3% of the time at top-5, where top-5 messages give you only 52.6% of it and thread expansion gives you all of it. A generic RAG hands you half the conversation, and it tends to be the half the answer is missing from.

just bench scores 360 committed queries against a fixed 2,000-document slice of public Enron-QA and prints recall@k with intervals and a paired test. Zero LLM calls, about 1.6 min on an Apple-silicon GPU. Method, caveats and every omission for both commands are in docs/BENCHMARK.md. Fixtures are committed under eval/demo/ and eval/public/.

以上為 README 的開頭部分。其餘內容請閱讀原文。 檢視完整 README ↗