PersonaQuery — Grounded RAG over Personal Documents
PersonaQuery is a retrieval-augmented generation (RAG) assistant that answers
questions about a single person (Rudra Joshi) strictly from that person's own
documents, with inline citations and prompt-injection guardrails.
It is a portfolio-scale system: a small private corpus, a hybrid retriever
built on local files, and a hosted LLM. See Scope & Limitations.
Architecture
data/private/*.pdf + data/curated/*.md
│ ingest (per-document chunk profiles)
▼
storage/ chunks.jsonl · vectors.npy · bm25.json · colbert_vecs.npz · faq_cache.json
query ─▶ guardrails ─▶ response cache ─▶ FAQ cache ─▶ query expansion
─▶ dense + BM25 (per variant) ─▶ RRF fusion ─▶ diversity cap
─▶ ColBERT rerank ─▶ context pack ─▶ Qwen3.8-27B ─▶ answer + citations
(extractive fallback if the LLM is down)
| Layer | Choice |
|---|
| Backend | FastAPI + Uvicorn (app/backend) |
| Frontends | Portfolio at https://rudrajoshi.dev (React) calls the backend; app/frontend-next is a separate Next.js frontend |
| Embeddings | BAAI/bge-small-en-v1.5 via FastEmbed (ONNX, no torch), 384-dim |
| Keyword | BM25 (rank-bm25) |
| Fusion | Reciprocal Rank Fusion (k=60) |
| Rerank | ColBERT late interaction (answerai-colbert-small-v1), vectors precomputed at ingest |
| LLM | qwen/qwen3.8-27b via Groq free tier (OpenAI-compatible), temperature 0.2 |
| Vector store | NumPy matrix + JSONL chunks; no external DB |
| Deployment | FastAPI backend on Google Cloud Run (Docker, app/backend/Dockerfile) |
Running locally
cd app/backend
pip install -r requirements.txt
echo "GROQ_API_KEY=gsk_..." > .env # free key from console.groq.com
python -m scripts.ingest # rebuild storage/ from data/
python -m scripts.precompute_faq # rebuild FAQ answers (calls the LLM)
python -m uvicorn src.main:app --reload # serves /chat and /health
python -m pytest tests -q # unit + integration tests
python -m src.eval.gate # retrieval-only gate (no token needed)
python -m src.eval.run_eval # full eval incl. generation
Key environment variables:
| Var | Default | Purpose |
|---|
GROQ_API_KEY | — | Groq API key (required for generation) |
GROQ_CHAT_MODEL | qwen/qwen3.8-27b | answer model |
GROQ_EXPANSION_MODEL | openai/gpt-oss-20b | query-expansion model (separate per-model rate limits) |
TOP_K | 8 | retrieved chunks |
QUERY_EXPANSION_ENABLED | true | LLM paraphrases for long questions |
RERANK_ENABLED | true | ColBERT rerank |
FAQ_CACHE_ENABLED | true | serve precomputed answers |
INJECTION_GUARD_ENABLED | true | prompt-injection guard |
Evaluation
src/eval/qa.jsonl is an 18-question golden set. CI runs the test suite and
fails if retrieval hit-rate@8 drops below 90%. Current: 94.4% (17/18).
The research paper and patent PDFs are deliberately not ingested (co-author
PII, reference-list noise); data/curated/projects.md summarises both.
Known limitations
- q10 ("Summarize my professional profile.") is borderline and currently
fails retrieval. "profile" matches the curated
linkedin-profile.md,
education.md and projects.md chunks about as well as the CV summary
line, which sits just outside the top 8 (it is 3rd at top-10). It has
flipped between pass and fail with ColBERT model and corpus changes. Left
failing rather than tuning the golden set to pass. 18 cases is a small set,
so treat the score as a regression alarm, not a guarantee.
Scope & Limitations
- Tiny, personal corpus (~58 chunks). Retrieval metrics would not transfer
unchanged to a large heterogeneous corpus.
- Brute-force vector search over an in-RAM matrix. Fine at this size.
- Single LLM provider. The FAQ cache and extractive fallback keep the bot
answering during a provider outage; there is no second provider.
- In-process rate limiter and caches: single-instance only.
- Eval is snippet/keyword based, not a semantic judge.
- No authentication. Single-persona, public read-only.
