Scouter

An interactive roofline lab for LLM training and inference, MoE and dense. Pick a model and a GPU, and Scouter estimates a decode step, a chunked prefill or a post-training step kernel by kernel, then places every attention and MLP module on the roofline.

Live: https://hecate0821.github.io/Scouter/ · 中文：README.zh.md

For the five models with public weights and a HuggingArch spec (Qwen3.5-35B-A3B, Kimi K3, GLM-5.3-Flash, DeepSeek-V4.1-Flash, Qwen3.8-27B), the realistic numbers come from HuggingArch (MIT, by Han Shen), a per-operator roofline simulator, run offline from a fork in vendor/HuggingArch. Scouter puts it next to its own ideal lower bound and breaks the gap down item by item (docs/hga.md).

Every number is an estimate derived from open-source implementations and public data. Use it to learn, not to plan capacity.

Quick start

git clone --recursive https://github.com/Hecate0821/Scouter.git   # the HuggingArch fork is a submodule; the page itself does not need it
python3 -m http.server 8000      # or just open index.html; no build step

node tools/scouter.js infer --model kimik3 --hw B200 --basis ideal --B 128 --pick stepT,bound

node tools/scouter.js hga --model qwen35 --engine sglang --sig tp8 --hw H200 --isl 8192 --b 64 --axis b

cd tools/hga && uv run python export.py --model qwen35      # re-export HuggingArch's simulations (needs the submodule)

cd tests && npm install && npm test

Three sources

The top-bar switch (and --basis on the CLI) picks where a number comes from:

Source	Meaning
Ideal lower bound	The time this configuration cannot beat: peak throughput, no overheads, only necessary traffic, and a step at least as long as the critical path and each resource's total. It stays below every measured calibration point. Conventions follow k3-inference-calculator, generalized to all models.
HuggingArch (default, real models)	HuggingArch's simulation: each operator is max(compute, HBM, communication, launch floor), operators add up serially, collectives are α-β with the engine's algorithm. Precomputed per engine signature (5 per engine, shared by every model: SGLang, vLLM, Megatron-Core, torchtitan) × GPU over a context × batch grid, interpolated live. The step pages open with a comparison against the ideal lower bound (waterfall, per-class time, resources, curves); pages HuggingArch does not cover show the ideal lower bound.
Modeled (default, edited / custom models)	Scouter's own estimate, used only where HuggingArch has no spec: per-kernel-class efficiency, launch and sync costs, expected expert coverage, load imbalance, and the collectives each engine actually uses.
Measured	Data points only: public numbers (with source links) and anonymized measurements.

Pages

Tab	What it shows
Overview	One decode step and one post-training step of the same model, side by side.
Model architecture	Layer stack, per-module roofline and knobs, mechanism diagrams, and all models compared.
Post-training	One step (fwd / dgrad / wgrad, recompute, comm, optimizer, memory), parallelism, low-precision recipes.
Inference	Decode / prefill, deployment and parallelism (recipe presets for attention TP × DP + wide EP, DCP and mixed chunked prefill, compared by computed throughput, concurrency cap and decode stall), low precision, speculative decoding, the KV cache (bytes per layer type, prefix hit rate of multi-turn agent sessions across HBM / host / remote tiers, how SGLang, vLLM, TRT-LLM and TokenSpeed manage it), and checks against the SGLang cookbook and your own profiler trace.
RL	Train/inference mismatch, deterministic rollout, low-precision rollout and QAT.
Other	Network: bandwidth per GPU from HBM to the spine, NVSwitch and NVLS, in-node PCIe and GPUDirect, PCIe-only machines, rails, oversubscription and ECMP; where training and inference time goes on a given fabric, bandwidth versus latency, NVLink domain size, PD / weight-sync transfers, and the model against public nccl-tests results. Profile: one inference step split the way a live trace splits it (kern's compute / comm / idle and the CUPTI HES post's categories), DP and EP load imbalance, the cost of always-on tracing, and the tools.
Appendix	Measured kernels vs. the rooflines, the efficiency table, public data points, and the modeling method.

Models: Qwen3.5-35B-A3B, Kimi K3, GLM-5.3-Flash, Muse Spark Open, DeepSeek-V4.1-Flash, Qwen3.8-27B (dense baseline). GPUs: H100, H200, B200, B300, GB200, GB300, RTX PRO 6000 Server. Topologies: HGX 8-GPU node, NVL72 rack (64 GPUs), 8-GPU PCIe-only server.

The URL keeps the context, e.g. ?model=kimik3&hw=B200&src=ideal#infer/step. UI text lives in locales/ (docs/i18n.md).

This is the beginning of the README. Read the rest at the original. Read the full README ↗