ResolveIQ

An AI-assisted support-ticket triage tool for a fintech company. A ticket comes in, gets classified and PII-redacted, relevant knowledge-base content is retrieved, a grounded draft reply is generated with cited sources and a confidence score, and a human agent approves, edits, or escalates it. Edits feed back into retrieval ranking, so a correction on one ticket measurably improves the draft on the next similar one — that feedback loop is the project's core mechanic (see 04_App_Flow.md).

Full spec docs: 01_PRD.md · 02_TRD.md · 03_UI_UX_Design.md · 04_App_Flow.md · 05_Backend_Schema.md · 06_Implementation_Plan.md · DATA.md (what's real training data vs. synthetic vs. authored, and why) · BUILD_SPEC.md (the phased post-launch hardening plan — bugs, tests, real data, deployment, UI, customer portal, benchmarking) · docs/checkpoints/ (an interview-prep writeup per phase: what was built, why, failure modes, and the honest answer to the hardest follow-up question)

Architecture

flowchart LR
    subgraph Pipeline["LangGraph pipeline (app/pipeline/graph.py), one run per ticket, async via Celery"]
        direction LR
        A[redact] --> B[classify]
        B --> C[breach_risk]
        C --> D[retrieve]
        D --> E[customer_context]
        E --> F[draft]
        F --> G[decide]
        G -->|specialist_flagged| H[specialist_review: wider retrieval + forced strong tier]
    end

    Ticket[POST /tickets] -->|status=new, enqueue| Pipeline
    Pipeline --> DB[(Postgres + pgvector)]
    F -->|urgency < threshold| Cheap[Cheap tier: LoRA-fine-tuned Qwen2.5-0.5B]
    F -->|urgency >= threshold| Strong[Strong tier: Groq-hosted gpt-oss-120b]
    Agent[Human agent: claim, then approve / edit / escalate] --> DB
    Agent -->|edit| Correction[correction_signals] --> Boost[retrieval_boost on kb_chunks] --> D

Four pipeline "agents" in the TRD's sense (distinct, inspectable steps — not autonomous free-acting agents): a trained classifier (category + urgency, not an LLM guess), a pgvector retrieval agent (category-filtered, correction-boosted), a drafting agent (cost-routed between a locally fine-tuned small model and a stronger hosted one), and a decision agent (confidence + urgency + category → review routing, never auto-send).

Features

1. Drafting + feedback loop (core, built deep) — classify → retrieve → draft → decide, human review, corrections re-rank retrieval.
2. Customer history context — a summary of a customer's prior tickets is injected into the (strong-tier) drafting prompt.
3. SLA breach-risk prioritization — a small classifier trained on a synthetic age/category/urgency/queue-depth rule reorders the queue.
4. Agent coaching insights — edit-distance, resolution time, and escalation rate aggregated per agent; pure reporting, no new backend logic.
5. Ticket claim/assignment — an agent claims a ticket (or is auto-claimed on resolve); a second agent gets a real 409, not a silent double-resolution, if they try to act on someone else's claimed ticket.
6. Specialist-review branch — a specialist-flagged ticket (low draft confidence, or a high-urgency dispute) gets a genuinely distinct second pass — wider retrieval, forced strong-tier drafting — before an agent ever sees it, not just a flag on the same draft a routine ticket got.

Production LLM techniques

Technique	Where it lives
Structured output	app/ml/classifier.py — trained classifier, not an LLM guess
Grounding + citations	app/drafting/draft.py — every draft cites the chunks it used
Prompt versioning	app/drafting/prompts.py — prompts as versioned functions, not inline strings
Model routing	app/drafting/draft.py — LoRA-fine-tuned small model for routine tickets, Groq-hosted model for harder/urgent ones
Retry + timeout + fallback	app/drafting/draft.py (tenacity retry + "needs manual response" fallback on exhaustion)
Guardrails (PII redaction)	app/redaction/pii.py — runs before any LLM call or log write
Observability/tracing	app/core/tracing.py — Langfuse @observe on every pipeline node + LLM/embedding call
Automated evaluation	app/eval/ragas_eval.py — RAGAS non-LLM context precision/recall on an 18-example held-out set
Feedback loop	app/retrieval/retrieve.py (adjust_retrieval_boost) — the project's one differentiating mechanic

Tech stack

FastAPI (async) · PostgreSQL + pgvector · LangGraph · Celery + Redis · Next.js · JWT auth (access + rotated refresh) · Langfuse · RAGAS · GitHub Actions CI. Rationale for each choice is in 02_TRD.md.

Quick start

Database and broker are cloud-hosted (Phase 4): Postgres+pgvector on Neon (serverless, free tier, branching), Redis on Upstash (serverless, free tier). Nothing to install locally for either — just two connection strings. The API, worker, and frontend still run as native processes on your machine (no Docker — Docker Desktop was unstable on this project's dev machine).

This is the beginning of the README. Read the rest at the original. Read the full README ↗