CostGrid
A meter and a shut-off valve for LLM API spend.
Your software calls an LLM API. CostGrid sits in front of that call: it meters
every request against the provider's own reported token counts, attributes the
cost to a team and an agent, enforces the budgets you set before the money is
spent, and, when you ask it to, serves the call on a cheaper model instead of
refusing it.
Deployment is one line of configuration. Point your SDK's base URL at CostGrid
and nothing else changes:
client = Anthropic(base_url="https://costgrid.example.com", api_key="unused")
Starting a trial? docs/ONBOARDING.md is the one to
read. It takes you from nothing to a real number, assumes nothing, and ends
with four weeks of what to do next.
Try the dashboard against sample traffic. No
install. Then docs/QUICKSTART.md takes a clean
checkout to a real spend report from your own traffic, and
docs/TRIAL.md covers what a free trial involves and what
data stays inside your network (all of it).
What it does
Meters exactly. Token counts come from each provider's usage object, not
an estimate or a tokenizer guess. Five buckets are priced independently: input,
output, 5-minute cache write, 1-hour cache write, and cache read. Money is
integer nanodollars end to end. Never a float, because a float rounds and a
bill has to reconcile. Rates for 58 models across Anthropic and OpenAI are
checked against the providers' published tables by npm run verify-pricing,
which fails the build on a discrepancy or a stale verification date.
Attributes. Two headers (x-costgrid-agent, x-costgrid-department) turn
a provider invoice into per-team, per-feature chargeback. Unlabelled traffic is
metered as unattributed. A visible cost line rather than a silent gap.
Enforces before spending. Budgets per tenant, department or agent; model
allowlists and denylists; output-token caps. Each rule runs in monitor
(record only), warn (allow, annotate) or block (refuse before forwarding, so
the call costs nothing).
Draws boundaries, not just budgets. costgrid policy deny-tool agent:support refund_customer stops that agent. And every agent it delegates
to. From being handed that tool. It holds because a model cannot call a tool it
was never given: the tool list is part of the request, so the request is
refused and there is nothing for your harness to execute. No change to your
agent framework, and no trusting the model to respect an instruction. The same
rule is checked again on the way back, for a model that invents a tool name it
was never offered. Airtight on a buffered reply, and on a streamed one it cuts
before the arguments are sent. docs/QUICKSTART.md states exactly how far each
half goes.
Degrades instead of breaking. A cap with --fallback downgrades
over-budget traffic to a cheaper model rather than returning 403, so the
customer's product keeps answering. It refuses to make a substitution it cannot
make safely. Across providers, or to a model it cannot price.
Realises the saving. A route rule sends matching traffic to a cheaper model
and records the counterfactual, so the saving is audited rather than claimed.
Always dry-run first: --action monitor changes nothing and still measures what
it would have saved.
Proposes its own rules, and proves them first. costgrid advise reads your
metered traffic, drafts concrete policies, and replays each one against the
same window. So a saving is shown before anything is switched on. It proposes
and never applies; every line it prints is a command for you to run, or not.
Proposals that save money are kept separate from ones that merely bound a risk,
because ranking both by "money involved" would put a spending cap that saves
nothing above a route rule that saves real money. No model is involved: every
figure comes from a deterministic query over your own rows.
Reports. A live dashboard, a terminal report, and a monthly statement.
Spend by team with movement against last month, budget status, and what routing
saved. Exportable as CSV for finance.
Imports history. Backfill from each provider's admin API, so the dashboard
is not empty on day one. Imported rows are kept apart from metered ones: they
are daily provider totals with no attribution, and folding them together would
misattribute them.
Repository contents
packages/
core/ Pure domain: money, pricing catalog, usage parsing, policy. No I/O.
db/ SQLite schema, repositories, read-side analytics, statements.
gateway/ The proxy: auth, enforcement, metering, dashboard, control plane.
cli/ Operator surface: keys, policies, reports, statements, backups.
desktop/ The app: the gateway and its console in a window, for one computer.
docs/ QUICKSTART (self-hosted), TRIAL (what a trial involves),
HOSTING (as a service), ARCHITECTURE (decisions and their reasons),
adr/ (architecture decision records).
index.html The product site. research.html lists the sources and the LCSO
method. site.css and site.js are shared by both pages.
demo/ The real dashboard against a captured snapshot, for GitHub Pages.
scripts/ Pricing verification and an end-to-end smoke test.
CostGrid_Token_Cost_Model.xlsx The forecast model, 2023 to 2030.
npm install && npm run build && npm test
Site: https://costgrid.dev ·
Dashboard demo: https://costgrid.dev/demo/ ·
Research and the LCSO method: https://costgrid.dev/research.html
The model behind it
CostGrid started from an observation: per-token prices are collapsing and
enterprise AI bills are rising anyway. Unit price is the wrong thing to watch.
Spend is price multiplied by volume, and volume usually wins. The research
page and the spreadsheet are that research, and the routing economics in
packages/core/src/routing.ts come straight from it.
This is the beginning of the README. Read the rest at the original. Read the full README ↗
