CostGrid

A meter and a shut-off valve for LLM API spend.

Your software calls an LLM API. CostGrid sits in front of that call: it meters every request against the provider's own reported token counts, attributes the cost to a team and an agent, enforces the budgets you set before the money is spent, and, when you ask it to, serves the call on a cheaper model instead of refusing it.

Deployment is one line of configuration. Point your SDK's base URL at CostGrid and nothing else changes:

client = Anthropic(base_url="https://costgrid.example.com", api_key="unused")

Starting a trial? docs/ONBOARDING.md is the one to read. It takes you from nothing to a real number, assumes nothing, and ends with four weeks of what to do next.

Try the dashboard against sample traffic. No install. Then docs/QUICKSTART.md takes a clean checkout to a real spend report from your own traffic, and docs/TRIAL.md covers what a free trial involves and what data stays inside your network (all of it).

---

What it does

Meters exactly. Token counts come from each provider's usage object, not an estimate or a tokenizer guess. Five buckets are priced independently: input, output, 5-minute cache write, 1-hour cache write, and cache read. Money is integer nanodollars end to end. Never a float, because a float rounds and a bill has to reconcile. Rates for 58 models across Anthropic and OpenAI are checked against the providers' published tables by npm run verify-pricing, which fails the build on a discrepancy or a stale verification date.

Attributes. Two headers (x-costgrid-agent, x-costgrid-department) turn a provider invoice into per-team, per-feature chargeback. Unlabelled traffic is metered as unattributed. A visible cost line rather than a silent gap.

Enforces before spending. Budgets per tenant, department or agent; model allowlists and denylists; output-token caps. Each rule runs in monitor (record only), warn (allow, annotate) or block (refuse before forwarding, so the call costs nothing).

Draws boundaries, not just budgets. costgrid policy deny-tool agent:support refund_customer stops that agent. And every agent it delegates to. From being handed that tool. It holds because a model cannot call a tool it was never given: the tool list is part of the request, so the request is refused and there is nothing for your harness to execute. No change to your agent framework, and no trusting the model to respect an instruction. The same rule is checked again on the way back, for a model that invents a tool name it was never offered. Airtight on a buffered reply, and on a streamed one it cuts before the arguments are sent. docs/QUICKSTART.md states exactly how far each half goes.

Degrades instead of breaking. A cap with --fallback downgrades over-budget traffic to a cheaper model rather than returning 403, so the customer's product keeps answering. It refuses to make a substitution it cannot make safely. Across providers, or to a model it cannot price.

Realises the saving. A route rule sends matching traffic to a cheaper model and records the counterfactual, so the saving is audited rather than claimed. Always dry-run first: --action monitor changes nothing and still measures what it would have saved.

Proposes its own rules, and proves them first. costgrid advise reads your metered traffic, drafts concrete policies, and replays each one against the same window. So a saving is shown before anything is switched on. It proposes and never applies; every line it prints is a command for you to run, or not. Proposals that save money are kept separate from ones that merely bound a risk, because ranking both by "money involved" would put a spending cap that saves nothing above a route rule that saves real money. No model is involved: every figure comes from a deterministic query over your own rows.

Reports. A live dashboard, a terminal report, and a monthly statement. Spend by team with movement against last month, budget status, and what routing saved. Exportable as CSV for finance.

Imports history. Backfill from each provider's admin API, so the dashboard is not empty on day one. Imported rows are kept apart from metered ones: they are daily provider totals with no attribution, and folding them together would misattribute them.

---

Repository contents

packages/
  core/     Pure domain: money, pricing catalog, usage parsing, policy. No I/O.
  db/       SQLite schema, repositories, read-side analytics, statements.
  gateway/  The proxy: auth, enforcement, metering, dashboard, control plane.
  cli/      Operator surface: keys, policies, reports, statements, backups.
  desktop/  The app: the gateway and its console in a window, for one computer.
docs/       QUICKSTART (self-hosted), TRIAL (what a trial involves),
            HOSTING (as a service), ARCHITECTURE (decisions and their reasons),
            adr/ (architecture decision records).
index.html  The product site. research.html lists the sources and the LCSO
            method. site.css and site.js are shared by both pages.
demo/       The real dashboard against a captured snapshot, for GitHub Pages.
scripts/    Pricing verification and an end-to-end smoke test.
CostGrid_Token_Cost_Model.xlsx   The forecast model, 2023 to 2030.

npm install && npm run build && npm test

Site: https://costgrid.dev · Dashboard demo: https://costgrid.dev/demo/ · Research and the LCSO method: https://costgrid.dev/research.html

---

The model behind it

CostGrid started from an observation: per-token prices are collapsing and enterprise AI bills are rising anyway. Unit price is the wrong thing to watch. Spend is price multiplied by volume, and volume usually wins. The research page and the spreadsheet are that research, and the routing economics in packages/core/src/routing.ts come straight from it.

This is the beginning of the README. Read the rest at the original. Read the full README ↗