benchbench
Same prompt. Every model.
An open gallery of AI-generated code. Every model gets the exact same request,
once per reasoning level. You see what it actually built, running live,
next to its code, its cost and how long it took.
No scores, no judges: you look and decide.
What's inside
| |
|---|
| 44 models | Claude, GPT, Gemini, Grok, DeepSeek, Qwen, Kimi, GLM, Mistral, MiniMax, Muse, Gemma, Nemotron… |
| 12 prompts | 6 categories, each with a precise prompt and a casual one-liner |
| 1,244 results | every reasoning level a model supports, from off to max |
| Everything kept | rendered demo, screenshot, source code, reasoning trace, raw response, tokens, cost, time, console errors |
The prompts
Each category has two prompts. The standard one is a precise list of requirements. The one-line idea is a single casual sentence, written the way a real person would ask.
| Category | Standard prompt | One-line idea |
|---|
| HTML/CSS/JS | Kanban board | Food truck website |
| Snake | Classic Snake | Train snake |
| Breakout | Classic Breakout | Hamburger breakout |
| Three.js | Forest campfire at night | Interactive 3D aquarium |
| SVG | Fox under the moon | Skateboarding astronaut cat |
| CSS only | Animated weather cards | Exploding like button |
All prompts share one system prompt. It asks for a single self-contained file that works in a sandboxed iframe, shows something on first paint and has no console errors. It also asks games to start in a self-playing demo mode, so you can judge them without playing.
How it works
prompts/*.md ──► generate.mjs ──► results/ ──► screenshot.mjs ──► manifest.json ──► site/ (Astro) ──► static site
(OpenRouter) code, metrics, (headless Chrome, a page per prompt,
reasoning trace console errors) per model, per result
- Generate.
scripts/generate.mjs sends every prompt to every model through OpenRouter, for each requested reasoning level. It streams the answer and keeps the code, the reasoning, the token counts, the real cost and the time.
- Check.
scripts/screenshot.mjs loads each demo in a sandboxed iframe at 1280×800, takes a screenshot after 3 seconds and records every console error.
- Publish.
site/ is an Astro site that reads results/manifest.json and builds a fully static gallery.
Generated code never runs on the site's own origin. Every demo is shown in an <iframe sandbox="allow-scripts">.
Run it yourself
You need Node 22.23 or newer (24 recommended), an OpenRouter key, and Chrome installed for the screenshots.
npm install
cp .env.example .env # then paste your OpenRouter key
npm run estimate # what would run, and roughly how much it would cost
npm run generate -- --models anthropic/claude-sonnet-5.5 --reasoning default,high
npm run shots # screenshots and console checks for the new results
npm run manifest # rebuild results/manifest.json
npm run site:dev # browse locally on http://localhost:4321
npm run site:deploy # build and publish to Cloudflare Workers
Useful options for generate:
| Option | What it does |
|---|
--models a,b | Models from bench.config.json, or any exact OpenRouter id |
--prompts snake,css-standard | Only some prompts, by category or id |
--reasoning default,none,low,high,max | Reasoning levels. An unsupported level falls back to the closest one |
--budget 5 | Stop starting new generations past this many dollars |
--timeout 45 | Minutes allowed per generation |
--force | Redo results that already exist |
--dry-run | List the work and estimate its cost, without calling any model |
Existing results are skipped, so a run that stops halfway can simply be started again. The script also caps itself at the credit left on the account and on the key, and stops cleanly when either runs out.
Hosting
The site is fully static. site/wrangler.jsonc serves site/dist with Cloudflare Workers static assets, with no Worker code. npm run site:deploy builds and publishes it, using a Wrangler login (npx wrangler login). Any static host works too: build with npm run site:build and upload site/dist.
Add a prompt
Create a Markdown file in prompts/<category>/:
---
id: snake-oneline
category: snake
level: oneline
title: Train snake
output: html
---
make me a snake game where the snake is a little train with a locomotive and colorful wagons
output is html or svg. Each result stores a hash of the system prompt and the prompt body. If either one changes, generate runs that prompt again. Changing only the title does not.
Repository layout
prompts/ the system prompt and the 12 prompts
scripts/ generate.mjs, screenshot.mjs
results/ one folder per prompt / model / reasoning level, plus manifest.json
site/ the Astro site
bench.config.json default models, budget, timeouts
License
MIT. The demos in results/ were written by the models listed next to them.
