AI Fit
Live: https://mkorbi.github.io/ai-fit/ (GitHub Pages) · source: https://github.com/mkorbi/ai-fit
A capacity planner for LLMs you host yourself. Pick accelerators, how many, the fabric inside a node and the
network between nodes, then a model and a workload, and it tells you:
- how many users (and concurrent requests) the hardware serves at a given context length,
- how much context each session can have at a given load,
- tokens per second per user, aggregate throughput, time to first token, cost per million tokens and power,
- which optimizations (KV cache and weight quantization, prefix caching, speculative decoding, session retention,
prefill/decode disaggregation, tensor vs pipeline layout, data-parallel attention and expert parallelism for
mixture-of-experts models, including expert-parallel groups across nodes) buy how much more room.
The reverse direction works too: give it users, context, model and targets and it sizes the smallest layout per
candidate accelerator, sorted by count and price.
Simulator
The Simulator tab is a pixel-art serving simulation on the same engine. A rack of GPUs fills with weights (blue),
runtime overhead and headroom (grey) and one coloured block per active person's KV cache; parked sessions sit in a
host-RAM strip or stay dimmed in the rack, depending on the retention setting. Pixel people think, wait for a free
slot, wait for their first token, stream tokens at the speed the engine predicts for the current number of active
people, and go back to thinking. Every knob is live (GPU type and count, model, weight and KV precision, context,
people, activity, wanted speed, retention, prefix caching, speculative decoding), presets give starting points, a
"what limits you" line names the binding constraint, and bar charts show how many people the setup serves if you
change one thing: context, GPU type, model, KV precision or weight precision. node tests/sim.js checks the presets
and the sweeps.
Buy or rent
Every accelerator carries a purchase price (buy in catalog.js: list or retail price, a share of a full system
price, or an analyst estimate, each with its source). providers.js holds on-demand prices per accelerator-hour at
AWS, Microsoft Azure and Google Cloud (USD) and at the European sovereign clouds Scaleway, STACKIT and OVHcloud (EUR,
excluding VAT, not converted), with the instance type and source of each. The planner's "Buy or rent" panel prices
the current layout at every provider, per month and per million tokens at your load, and shows after how many months
buying would pay off (purchase price only, without power, hosting or staff). The Catalog lists the whole matrix.
node tests/prices.js checks that every price points at a known accelerator and has a source.
Engine stack
The Engine stack tab lays out the software around the accelerators for inference serving, fine-tuning and model
training, with vLLM as the reference engine: twelve layers from drivers to identity, which purpose needs which, the
options per layer with licences and links, the reference pick, and what each layer needs in CPU, RAM and disk, split
into services beside the GPUs and the share on the GPU servers. The content lives in stack-data.js;
node tests/stack.js checks it is complete.
Run it
Open index.html in a browser. Everything is client-side; the only network requests are the Google Fonts.
Inputs and custom catalog entries are kept in the browser's local storage.
Host it on GitHub Pages
The folder is a static site with relative paths, so it works from a repository root or any subfolder:
- Push the folder to a GitHub repository (
.nojekyll is included so Pages serves the files untouched).
- In the repository: Settings → Pages → Build and deployment → Source "Deploy from a branch",
branch
main, folder / (root) (or /docs if you put the files there).
- The site appears at
https://<user>.github.io/<repo>/ after the first deploy, usually within a minute. This copy is served at https://mkorbi.github.io/ai-fit/.
Publish as a Claude artifact
The artifact publisher wraps a page in its own document skeleton, so it takes the body-only variant
artifact.html. Regenerate it from index.html after every change:
sed '1,/^<body>$/d; /^<\/body>$/,$d' index.html > artifact.html
以上为 README 的开头部分。其余内容请阅读原文。 查看完整 README ↗
