AI-Native Piano Learning RAG Assistant

A retrieval-augmented generation (RAG) assistant over a piano-learning knowledge base, built in three phases: an offline ingestion pipeline, a query-time assistant, and free off-localhost hosting with built-in observability. Everything runs on free/local, open-source components by default; the only optional hosted piece (the LLM in Phase 3) uses a free tier — no paid API anywhere. Built as part of My AI Portfolio alongside the AI-Native Team Diagnostic.

---

Try it live

🎹 Open the assistant →

A static GitHub Pages chat front end calling a Google Cloud Run backend (Express + local retrieval/rerank) with Groq doing the reasoning and Neon holding the vectors — all on free tiers, $0.

• Ask things like "How do I build finger independence?" or "What exercises help weak 4th and 5th fingers?"
• Every answer cites its sources (book + page) and shows a live latency/tokens/cost line.
• Heads-up: the backend scales to zero, so the first request after idle wakes it (~15–30 s). The page pre-warms it on load and shows a "waking up…" spinner with auto-retry, so a cold start is just a slower first answer, not an error; subsequent requests are fast.

---

Executive Summary

Problem. Piano method books and exercise references (Chang's Fundamentals of Piano Practice, Hanon's Virtuoso Pianist) are long, unindexed, and don't map to how a learner actually asks questions — "what should I do about a weak 4th finger" doesn't correspond to any table of contents entry. This project builds a full RAG system to solve that retrieval problem end-to-end, deliberately architected the way an enterprise knowledge-base assistant would be: hybrid search, reranking, citations, guardrails, and production observability — not a toy demo. The piano corpus is the test case; the pattern (turn a large, domain-specific document set into a cited, guarded Q&A service) is the same shape as an internal enterprise KB over policy docs, runbooks, or onboarding material.

User. Primary (as built): a self-directed piano learner asking practice and technique questions in natural language and getting a cited, grounded answer instead of a manual search through 200+ pages. Enterprise analogue: a new employee or support agent querying an internal knowledge base instead of keyword-searching a wiki — same retrieval and trust problem (does the answer cite the right source, and does the system know when to say "I don't know"), different corpus.

Objective. Build a production-shaped RAG architecture — hybrid retrieval, reranking, cited generation, and refusal guardrails — on a real, non-trivial corpus, with the discipline of a system built to be trusted, not just demoed. The piano knowledge base is the test bed; the actual objective is showing the pattern works end-to-end: ingest a real document set, retrieve accurately, generate only from what's retrieved, and know when to say "I don't know." That pattern is what would transfer to an enterprise KB, not the piano content itself.

Enterprise applicability. What's being demonstrated is the operational lifecycle of a production RAG platform, exercised end-to-end — ingestion, hybrid retrieval, evaluation (a deterministic LLM-judge), observability (per-request tracing), deployment (free-tier, off-localhost), governance (cited answers, refusal guardrails, source attribution), and continuous improvement (nightly eval + benchmark). The piano books are just the test corpus; that lifecycle is the transferable part.

Success metric. Two tiers, deliberately kept separate:

• System health — measured live, per request, via built-in observability persisted to a searchable Neon table.
  • Tracked: latency (p50/p95), token cost, and "grounded rate" (does every answer carry a citation).
  • Current: $0 LLM cost (Groq free tier); grounded rate (~82% — the rest are honest refusals, since the benchmark deliberately includes thin-coverage / out-of-scope questions the assistant declines) and latency tracked live — see Live observability.
  • Made statistically meaningful: system-health metrics are now measured over 122 accumulated automated benchmark queries (npm run bench, sampled from a 120-question bank, run nightly until target) — a large enough sample for latency / grounded rate / cost to be statistically meaningful, not a handful of hits.
  • Kept honest: benchmark traffic is tagged and counted separately from organic live queries — test traffic by design, not a claim of real-world usage.
• Retrieval quality — because "grounded" only confirms an answer has a citation, not that it cites the right passage.
  • First-pass (keyword eval, npm run eval): Hit@5 100% (8/8) · MRR 0.938 · out-of-scope refusal 100% (2/2).
  • Caveat: relevance is a keyword proxy over 10 questions — a regression signal, not human-judged ground truth.
  • Deeper (Phase 4 LLM-judge, npm run eval:judge): a deterministic judge scores answer correctness, hallucination rate, and a semantic Hit@5 over a larger golden set — see Phase 4.

Acceptance criteria (retrieval quality). Concrete, checkable targets — each with how it's verified and where the automated test lives:

This is the beginning of the README. Read the rest at the original. Read the full README ↗