MarginMaestro
An agentic, event-driven platform that automates the end-to-end margin call lifecycle — from market event to client notification, escalation, and audit — using LLM agent orchestration, a RAG pipeline over legal/policy documents, and a real-time streaming backbone.
Status: ✅ Live on Google Cloud. API, MCP servers and event consumers on Cloud Run; Cloud SQL Postgres + pgvector; Gemini on Vertex AI; the "Ask the Desk" ADK agent on Agent Platform; frontend on Firebase App Hosting + Vercel. The project was first built and fully deployed as a standalone production stack on AWS — EC2, Secrets Manager, S3, IAM — with Azure SQL and OpenAI driving the same agent pipeline, proving the architecture end to end before it was adapted to the GCP-native services below. See docs/gcp/GCP_ROADMAP.md and docs/gcp/GCP_PROGRESS.md for the GCP track, docs/ROADMAP.md / docs/PROGRESS.md for the original AWS-era phases.
🚀 Try the live demo
This is the Google Cloud deployment (the hackathon submission) — everything below runs on Cloud Run, Cloud SQL and Vertex AI, not AWS.
Cloud Run scales to zero when idle, so the first request after a quiet period can take 10-20s to cold-start — that's expected, not a hang.
Sign in at /login with one of the seeded demo accounts (also the default values of demo_approver_password / demo_manager_password / demo_analyst_password / demo_auditor_password in src/config/settings.py — nothing sensitive, these exist purely to gate the demo dashboard):
| Username | Password | Role | Can do |
|---|
approver | MarginMaestro!Approver1 | Approver | Approve / reject / adjust a margin call; respond to its SLA |
manager | MarginMaestro!Manager1 | Manager | Everything approver can, plus the required second sign-off on elite-tier counterparties |
analyst1 | MarginMaestro!Analyst1 | Margin analyst (viewer) | Read-only, own book only: CP-1 … CP-4 |
analyst2 | MarginMaestro!Analyst1 | Margin analyst (viewer) | Read-only, own book only: CP-5 … CP-8 |
auditor | MarginMaestro!Auditor1 | Auditor | Read-only, every counterparty incl. system-level audit rows |
Every page except the public landing page needs a login, and the backend scopes every read to the caller: on Postgres, row-level security in the database itself hides other analysts' counterparties (MM-106). The approver, manager and auditor see all counterparties.
This is a portfolio demo running on the project owner's own Google Cloud/Slack/WhatsApp accounts — please don't script/load-test it. A handful of clicks is exactly what it's for.
A five-minute walkthrough
- Simulate Event — pick a counterparty and a market shock (e.g. a price move on one of the curated tickers) and fire it. This kicks off a real LangGraph run: exposure is recomputed, the counterparty's CSA is retrieved via RAG, and a breach is evaluated.
- Approvals & SLA — if the shock breached the threshold, a margin call is now waiting here. Approve it as
approver (elite-tier counterparties additionally need manager's second sign-off before the client notice goes out).
- Once approved, a real Slack message goes out and an SLA timer starts. Use Simulate counterparty response on the same tab to resolve it (met → a confirmation is posted back to Slack; left to expire → it escalates to a real ServiceNow incident).
- Agent Trace — pick the run you just triggered and watch its full step-by-step lifecycle (every LangGraph node, in order, with real timestamps and outputs) as a horizontal timeline.
- Margin Calls / Positions & Exposure — see the resulting call and updated exposure for that counterparty.
Prefer to watch it happen end-to-end without clicking? Clone the repo and run the same scenario the API itself uses for demos:
pip install -e ".[dev]"
python -m demo.run_demo --base-url https://marginmaestro-api-pkjc2r5ivq-uc.a.run.app
Why this project exists
In capital markets, a margin call is the process of demanding additional collateral when market moves erode the coverage on a portfolio of derivatives, repo, or financed positions. The call itself is simple arithmetic — but everything around it is not: interpreting bespoke legal agreements (CSAs), reconciling disputed valuations, choosing which collateral to post, notifying the counterparty, chasing non-response, escalating, and keeping a defensible audit trail.
Historically this is done with spreadsheets, email, and phone calls, and it is dangerously blind to the future — firms can tell you today's call but struggle to anticipate tomorrow's, which is exactly what caused margin-driven blow-ups in 2008, March 2020, the 2022 UK LDI crisis, Archegos, and LME nickel.
MarginMaestro models the full lifecycle as a mesh of specialized AI agents coordinated by an orchestrator, reacting to market events in real time, grounding its decisions in the actual legal and policy documents via RAG, and keeping a human in the loop for the decisions that matter.
What it demonstrates
- Agent orchestration — an orchestrator agent conducting specialist agents (event detection, calculation, CSA interpretation, dispute, collateral optimization, communication).
- RAG pipeline — retrieval over CSAs, margin policy, exception rules, escalation procedures, and historical dispute notes.
- Real-time streaming — a Kafka event backbone driving intraday, tick-level margin evaluation, with a pluggable real-vs-simulated market feed.
- Production engineering — containerized services, CI/CD with quality gates and code coverage, security scanning (CodeQL, Dependabot, secret scanning with push protection), IaC, secrets management, observability, and a full audit trail.
- Human-in-the-loop — approval gates (with a second sign-off for elite-tier counterparties) and SLA-driven escalation to a real ServiceNow incident, reflecting how regulated institutions actually operate.
High-level architecture
docs/architecture/functional-architecture.svg — the business-process view, in the same visual language as the tech diagrams below: detect & evaluate → decide (approval + second sign-off) → notify (WhatsApp + Slack) → respond & escalate (SLA timer → ServiceNow) → transparency (audit, lineage, reporting), with the actual touchpoints (WhatsApp, Slack, ServiceNow, Gemini, Firebase) called out where a real product is used.docs/architecture/functional-lifecycle.svg — the same lifecycle at the implementation level: every LangGraph node from compute_exposure through approval, notification, and SLA/escalation, color-coded by CLAUDE.md's golden rule (deterministic code vs. LLM reasoning/RAG vs. hybrid vs. the human-approval gate).docs/architecture/tech-architecture.svg — the original AWS deployment the project was first built and fully proven on: AWS (EC2 + Elastic IP, Secrets Manager, S3, IAM), Vercel, Azure SQL, the CI/CD pipeline, and the third-party integrations (OpenAI, Slack, ServiceNow), with a clearly separated box for what's local-dev-only (Kafka/Redpanda, OTel/Prometheus/Grafana) and not part of either deployment.docs/architecture/gcp-tech-architecture-blue.svg and gcp-tech-architecture-multicolor.svg — the live Google Cloud architecture (the GCP track in docs/gcp/GCP_ROADMAP.md): Cloud Run services (API, MCP tool servers private to the desk agent), Firebase App Hosting + Firestore live status, Pub/Sub and Cloud Tasks, Gemini on Vertex AI with Agent Platform (Sessions, Memory Bank, Gen AI evaluation in CI), Model Armor and Sensitive Data Protection, Cloud SQL + pgvector with row-level security, BigQuery + Dataplex + Data Lineage, the security/operations layer (including CodeQL/Dependabot/SonarCloud/Trivy/Checkov/gitleaks in CI), WhatsApp/Slack/ServiceNow, and the GitHub Actions → Docker Hub → Cloud Run (keyless WIF) deploy path. Same diagram in two icon styles: Google Cloud's official blue product icons, and its official four-colour core/category icons. PNG copies sit alongside each SVG.
See docs/ARCHITECTURE.md for the full written design, and docs/AGENT_ORCHESTRATION_FAQ.md for which parts are deterministic code vs. LLM-driven, and what happens end-to-end when a real (not simulated) market move triggers a run.
Tech stack
| Layer | Choice | Notes |
|---|
| Agent orchestration | LangGraph (margin-call workflow) + Google ADK (desk assistant) | Fixed, auditable state-graph for the lifecycle on Cloud Run; a conversational ADK agent on Agent Platform for analyst Q&A (ADR-0019) |
| LLM | Gemini on Vertex AI | Reasoning, retrieval and drafting only, never math; every call screened by Model Armor + in-code guardrails, with Sensitive Data Protection masking (OpenAI remains a supported fallback via LLM_PROVIDER) |
| Embeddings | Vertex AI text embeddings | Query/document embeddings share one model |
| RAG + relational store | Cloud SQL Postgres + pgvector | Positions, ratings, calls, audit log and RAG chunks in one database, with row-level security scoping every read to the analyst's counterparties; LangGraph checkpoints persisted here |
| Documents | Cloud Storage | CSA, policy, dispute and escalation documents (30-day retention, scanned by SDP) |
| Eventing | Pub/Sub + Cloud Tasks + Cloud Scheduler | Live price refresh, impact events, one SLA timer per call, the daily margin run |
| API | FastAPI on Cloud Run | Scales to zero; IAM database login; OIDC for internal callers |
| Frontend | Next.js on Firebase App Hosting + Vercel | Real-time ops dashboard and the "Ask the Desk" chat; the same app on both hosts against the same API, each git-linked to auto-deploy on push to main (App Hosting: managed Cloud Run, scales to zero; MM-145, ADR-0021) |
| Real-time status | Firestore + Firebase Auth | One small status doc per margin call, pushed to every open Approvals page and dashboard the moment a call changes; per-user scope via custom-token claims and security rules; Postgres stays the source of truth (MM-146, ADR-0021) |
| Agent platform | Vertex AI Agent Runtime | The desk assistant: Sessions + Memory Bank, min_instances=0; evaluated with Gen AI evaluation in CI |
| Tool interface | MCP servers on Cloud Run |
Repository layout
MarginMaestro/
├── README.md
├── CLAUDE.md # Agent operating guide (read first, every session)
├── CONTRIBUTING.md
├── TESTING.md
├── .env.example
├── .gitignore
├── src/ # Backend: agents, calc, streaming, rag, api, mcp_servers, ports/adapters,
│ # config, persistence, governance, warehouse, realtime, observability, ops
├── frontend/ # Next.js dashboard + "Ask the Desk" chat (Firebase App Hosting + Vercel)
├── tests/ # Unit, contract and integration tests (coverage gate ≥ 80%)
├── migrations/ # Alembic migrations (dual-dialect: SQL Server + Postgres)
├── infra/ # Terraform: gcp/ (live Google Cloud stack) + the original AWS stack,
│ # Prometheus/Grafana provisioning
├── firebase/ # Firestore security rules (real-time call status, MM-146)
├── tableau/ # G7 reporting workbook (Margin_Dashboard.twb) + its screenshots
├── scripts/ # One-off ops scripts (Cloud SQL bootstrap, RAG ingest, secret migration)
└── docs/
├── ARCHITECTURE.md # Lifecycle, agent mesh, streaming, data flow
├── AGENTS.md # Each agent: responsibility, IO, tools
├── DATA_SOURCES.md # Structured vs unstructured data map + free sources
├── ROADMAP.md # Original (pre-GCP) phased plan, mapped to Jira epics/stories
├── PROGRESS.md # Original-track living handoff log
├── AGENT_ORCHESTRATION_FAQ.md # Code vs. LLM breakdown; real vs. simulated trigger parity
├── data_catalog.yaml # Governance catalog: every table/document family, class, owner (G8)
├── architecture/ # Functional + technical architecture diagrams (see above)
├── assets/ # Hackathon deck screenshots + brand assets
├── warehouse/ # BigQuery warehouse guide + Tableau workbook guide (G7)
├── hackathon/ # AI Builder Cup submission materials
├── adr/ # Architecture Decision Records (original stack, ADR-0001…0007)
└── gcp/ # The live GCP track
├── GCP_ROADMAP.md # Phases G0–G10, service catalog, Jira mapping
├── GCP_PROGRESS.md # GCP-track living handoff log — read this first for current state
└── adr/ # GCP-specific ADRs (ADR-0008…0021)
Getting started (local)
The fastest way to see the app is the live demo above — this section is for running your own copy.
git clone https://github.com/AdarshMurali/MarginMaestro.git
cd MarginMaestro
cp .env.example .env # fill in OPENAI_API_KEY at minimum; see file for the rest
pip install -e ".[dev]"
docker compose up -d sqlserver redpanda chroma # skip the `app` service -- run the API locally instead, below
alembic upgrade head
python -m persistence.seed_users # seeds approver / viewer / manager (see table above)
python -m persistence.batch_loader # seeds counterparties, positions, CSA/RAG documents
uvicorn api.main:app --reload
Then, separately:
cd frontend
cp .env.example .env.local # points at your local API + NextAuth config
npm install
npm run dev # http://localhost:3000
make test / make lint run the backend test suite and linters (see CONTRIBUTING.md for the full contributor workflow). Frontend checks: npx tsc --noEmit and npx eslint . from frontend/.
Disclaimer
MarginMaestro is a portfolio / proof-of-concept project. It uses free and synthetic data, and its margin calculations (VM/IM/SIMM) are directionally correct approximations for demonstration, not production-grade risk models. It is not affiliated with any employer and uses no proprietary data.
