AuditFlow

Autonomous audit-evidence review for the Office of the CFO.

AuditFlow turns controlled financial-document bundles into review-ready audit workpapers. Five specialized agents cross-reference evidence, identify bounded exceptions, route them to a reviewer, and compile the result into a single workpaper.

Built for Syndicate by Maximor — AO Hackathon, Track 2: Autonomous Office of the CFO.

🚀 Live Demo · Backend API · Demo Video

The deployed demo runs the real five-agent pipeline rather than a static frontend mock.

Why AuditFlow?

Audit review is often slowed less by the final judgment than by collecting evidence, checking whether records agree, and documenting exceptions.

• 5 specialized AI agents with bounded responsibilities
• 14 controlled benchmark cases spanning clean, missing, conflicting, and multi-issue evidence
• Evidence-backed human review instead of silently accepting exceptions
• Real document upload with Claude's native PDF/image understanding, alongside the benchmark suite
• Second-pass evidence resolution on ambiguous cases, with disagreement surfaced rather than silently resolved
• Persistent reviewer decisions, building a record of confirmed vs. overturned findings

Benchmark results

Most benchmark cases pass on every run. Missing purchase order and missing receipt findings are produced by code, not by the model. A few cases have varied between runs in earlier testing (for example CASE_06, where the Evidence stage occasionally returned nothing). A full re-measurement on the current code has not been done.

Product Preview

CASE_09 · complex exception routed to review

CASE_01 · clean transaction auto-cleared

Controlled benchmark bundle selector

Five-agent pipeline

Intake → Evidence → Anomaly → Decision → Workpaper

Agent	Responsibility
Intake Agent	Classifies source documents and extracts structured fields.
Evidence Agent	Links transaction evidence and identifies missing support.
Anomaly Agent	Checks only the bounded exception taxonomy.
Decision Agent	Routes any finding to human_review; routes no findings to auto_clear.
Workpaper Agent	Produces the evidence-backed, review-ready workpaper.

The bounded exception types are duplicate invoice, amount mismatch, missing PO, missing receipt, vendor mismatch, date inconsistency, currency mismatch, and tax mismatch.

Evidence Resolver

On transactions where the Evidence Agent's first pass lands below a confidence threshold, a second independent pass runs and the two are compared. If they agree, the higher-confidence result is kept. If they disagree, both sets of findings are unioned rather than one being silently picked, and the disagreement is surfaced in the UI. This targets low-confidence ambiguity; it does not catch confidently wrong Evidence results (see the Known limitations section below).

Architecture and model strategy

AO (Agent Orchestrator) coordinates the pipeline; a FastAPI backend exposes the full run via POST /run-case (benchmark cases) and POST /run-case-upload (real documents). Anthropic Claude is used with a focused mixed-model strategy.

The table distinguishes code defaults from the live configuration reported by the server.

Agent	Code default (without model overrides)	Live model reported on 24 October 2026	Live effort
Intake	claude-sonnet-5	claude-sonnet-5	API default; no effort set
Evidence	claude-sonnet-5	claude-sonnet-5-5	medium
Anomaly	claude-opus-5	claude-opus-5-5	medium
Decision	claude-sonnet-5	claude-sonnet-5-5	medium
Workpaper	claude-sonnet-5	claude-sonnet-5-5	medium

The live assignments were reported from server logs at 11:36 IST on 24 October 2026. Render environment variables override code defaults where configured. Live assignments are a dated snapshot and may change.

The Anomaly Agent uses the stronger model because exception classification is the most nuanced reasoning step.

Benchmark and review controls

The benchmark contains 14 controlled scenarios: clean cases, duplicate invoices, amount and currency mismatches, tax calculation errors, missing POs and receipts, vendor mismatches, date inconsistencies, multi-issue cases, and incomplete or conflicting evidence. These are evaluation fixtures—not customer audits.

Known limitations: The benchmark contains 14 controlled synthetic scenarios, not customer audits. The earlier model-comparison run was stopped at 47 of 56 planned runs to conserve credits; a separate six-run rehearsal was saved separately. Each case/configuration in the 47-run comparison had at most one completed run, so that comparison does not establish repeatability on the current code. Missing purchase-order and missing-receipt findings are now produced by code rather than the model, but a full re-measurement of the current code has not been done. Some cases varied in earlier testing, including CASE_06, where the Evidence stage occasionally returned nothing.

Detected exceptions are routed to a reviewer, who can inspect the finding, evidence, confidence, rationale, and workpaper context. The current UI supports Clear exception, Request evidence, and Escalate actions, and every decision is persisted server-side (POST /feedback) and viewable in a decision history dashboard (GET /feedback/history).

Upload Documents

Real PDF and image documents can be uploaded and run through the full five-agent pipeline via POST /run-case-upload, using Claude's native document understanding for extraction — no separate OCR step. The 14-case benchmark remains available separately for reproducible evaluation.

Stack and deployment

Area	Implementation
Frontend	Next.js, React, TypeScript
API	Python, FastAPI, Uvicorn
Reasoning	Anthropic Claude Sonnet 5 and Claude Opus 5
Orchestration	AO (Agent Orchestrator)
Deployment	Vercel frontend, Render backend

Run locally

Prerequisites: Python 3.13.5, Node.js, and ANTHROPIC_API_KEY.

git clone https://github.com/abhijitmanna912001/auditflow.git
cd auditflow
python3.13 -m venv venv && source venv/bin/activate
pip install -r requirements.txt

If pip install fails with externally-managed-environment, you're on a system Python (e.g. Homebrew) that blocks global installs by design — the venv step above avoids this.

# Backend
ANTHROPIC_API_KEY=your-key uvicorn orchestration.api:app --host 127.0.0.1 --port 8000

# Frontend, in a second terminal
cd frontend
npm install
npm run dev

The API defaults to http://127.0.0.1:8000; set NEXT_PUBLIC_API_BASE_URL to point the frontend at another backend.

Google Drive (optional): setting NEXT_PUBLIC_GOOGLE_CLIENT_ID, NEXT_PUBLIC_GOOGLE_API_KEY and NEXT_PUBLIC_GOOGLE_APP_ID in frontend/.env.local (see frontend/.env.example) turns on the Drive features; with any of them empty, no Drive control appears. After clicking Connect Google Drive in the header:

• Select from Google Drive adds PDF or image files from Drive to a check, alongside local uploads.
• Save to Google Drive appears next to Save PDF report once a check has completed. It creates the PDF report in the browser and saves it as a new file in My Drive / AuditFlow / <Company> / <YYYY-MM>. Company is the Company name field for uploaded documents, or "No company named" when that is blank and for sample cases. An existing file is never overwritten: saving again gives a name ending in (2), (3) and so on. Save PDF report still opens the browser's print window.

Drive uses the drive.file scope, so AuditFlow only sees files and folders it created or that were selected in it. The access token stays in the tab's memory: it is never stored, never put in a URL and never sent to the backend, and saving to Drive makes no backend request or model call. Open the app at http://localhost:3000, the origin registered for the OAuth client. The manual checks with a real Google account are in docs/testing/drive-report-save-stage3-manual-test.md (not yet run).

Frontend checks: npm run typecheck, npm test and npm run build in frontend/.

API

POST /run-case
Content-Type: application/json

{ "case_id": "CASE_09" }

The response is the Workpaper Agent's review-ready JSON output.

POST /run-case-upload?case_id=<label>&use_resolver=true
Content-Type: multipart/form-data

files: one or more PDF/PNG/JPG/JPEG/WEBP files

Same response shape as /run-case, run against real uploaded documents. use_resolver=true routes Evidence through the second-pass resolver.

POST /feedback
Content-Type: application/json

{ "case_id": "CASE_09", "document": "INV-1009-A", "finding": "...", "agent_action": "human_review", "decision": "confirmed" }

Persists a reviewer decision (confirmed, overturned, or evidence_requested) tied to a specific finding.

GET /feedback/history

Returns every persisted reviewer decision, oldest first.

Repository map

agents/         Five agent implementations and unit tests
orchestration/  FastAPI pipeline endpoint
dataset/        14 ground-truth cases and text document fixtures
evaluation/     Benchmark scoring and report comparison
frontend/       Next.js reviewer workspace
docs/           Agent contract and product screenshots

Next steps

• Automated regression testing using the persisted reviewer feedback (overturned findings as a regression set)
• Workpaper export and broader audit-domain coverage
• Additional anomaly types beyond the current eight

Team

• Abhijit Manna — agent logic, orchestration, evaluation, backend reliability, deployment
• Garvit Mathur — dataset, frontend, testing, integration, observability, deployment preparation

Built for Syndicate by Maximor — AO Hackathon · Track 2: Autonomous Office of the CFO.