Care-Beacon Medical RAG System
A production Retrieval-Augmented Generation (RAG) system that answers patient questions about cancer from articles published by BC Cancer, the Canadian Cancer Society and Cleveland Clinic. Every answer cites the source paragraphs it was built from.
It offers two ways to ask:
- Chat (default): a multi-turn conversation with memory. Follow-up questions such as "What does that involve?" are rewritten into standalone searches, answers stream token by token, and every conversation is saved so it can be reopened later.
- Search: a single question with a single cited answer, cached in Redis.
Live: https://care-beacon-health.vercel.app
Companion repository: Vital-Atlas is the
upstream scraper that produces this system's corpus. Care-Beacon consumes Markdown and
serves answers; it does no crawling of its own. See
Where the corpus comes from.
Highlights
- Conversational RAG with memory. Follow-up questions are rewritten using the conversation, retrieval runs on the rewritten query, and the answer is generated with a character-budgeted history. Threads and messages persist in Postgres (Neon).
- Streaming on serverless.
POST /api/v1/chat streams Server-Sent Events from a Python function on Vercel: progress stages (rewriting → searching → ranking), then citations, then answer tokens. The answer is saved even when the client disconnects mid-stream; a partial answer is stored as interrupted.
- Measured, not assumed. A Langfuse evaluation harness with LLM judges gates each release. The latest gate: 19/19 safety refusals in multi-turn drift threads, 0.88 follow-up resolution, 0.96 grounded answers, about $0.0007 per chat turn (results).
- Cost and abuse controls. A daily LLM spend cap ($2/day by default), counted in Redis across all OpenAI calls; a per-IP rate limit at the Vercel firewall; admin endpoints behind an API key.
- Privacy by design. No accounts: each browser holds an anonymous owner key, and traces carry only its SHA-256 pseudonym. Clients get generic error messages, and logs record only the error type, never database error text.
- Tested end to end. 407 API tests, 41 web tests and 32 evaluation-tooling tests, run in GitHub Actions on every push and pull request.
Architecture
Click the diagram to open the interactive version: zoom, search, and links to the source code.
- Two Vercel services behind one route table:
/api/* goes to the FastAPI api service; everything else goes to the Next.js web service.
- Cited answers: the answer generator checks Redis first, then retrieves passages from Qdrant Cloud and writes the answer with OpenAI
gpt-4o-mini.
- Conversation storage: chat threads and messages live in Neon Postgres, reached through its pooled connection string. Migrations are plain SQL files in
api/migrations/.
- Observability: optional Langfuse tracing records each step of a turn (rewrite, search, rerank, generation) with token cost.
- Offline ingestion:
api/scripts/ingest.py runs on a local machine and loads the Vital-Atlas corpus into Qdrant. The deployed API has no ingestion endpoints.
How a question is answered
Click the diagram to open the interactive version: zoom, search, and links to the source code.
- Cache check. Redis is keyed on a SHA-256 hash of the normalized question, filters, and limits. A hit returns immediately, with no OpenAI calls.
- Wide retrieval. On a miss, the question is embedded with
text-embedding-3-small (1536 dimensions) and Qdrant returns 6× the requested number of chunks.
- Rerank and filter. A loose pre-filter keeps scores ≥ 0.3,
gpt-4o-mini reranks every remaining candidate, and every chunk scoring ≥ 0.6 is kept, with no top-k cut. (0.6 is the configured default; a request can set its own min_similarity.)
- Generate and cache.
gpt-4o-mini writes an answer with paragraph-level citations, cached for 1 hour. If no chunk passes the threshold, a fixed "no context" reply is returned without an LLM call, and it isn't cached.
How a chat turn is answered
Chat reuses the same retrieval and reranking as Search, with three additions: query rewriting, conversation history and streaming.
Click the diagram to open the interactive version: zoom, search, and links to the source code.
- Rewrite. On a follow-up,
gpt-4o-mini turns the last few turns plus the new message into one standalone search query, so "Does it reach cells all over the body?" searches for the treatment being discussed. The rewritten query is shown to the user as the search runs.
- Retrieve and rerank. The query goes through the same embed → Qdrant → LLM rerank pipeline as Search, filtered by the thread's own source and similarity settings.
- Generate with history. The most recent turns that fit a 6,000-character budget are included, with old
[n] citation markers stripped so they can't be confused with this turn's sources. Safety instructions are placed next to the question, where they hold up under multi-turn pressure.
- Persist. The user's message is saved before retrieval starts and the answer, with its citations and cost, once it completes. Changing a thread's settings adds a settings event to the conversation and applies to later turns only.
Each diagram opens in an interactive viewer hosted on the live site, where you can zoom, search, focus on a component, and follow it to the source lines it was drawn from. The diagram sources live in docs/diagrams/.
Project Status
Current Version: 4.0.0 - Live in Production (Vercel)
Build history
This is the beginning of the README. Read the rest at the original. Read the full README ↗
