SkillArena
See what a UI skill builds before you install it.
SkillArena gives every UI skill (a SKILL.md for AI coding agents such as Claude Code, Cursor or Codex) the same product briefs and the same model, and shows the generated pages next to a no-skill control group. Browse it at skillarena.dev.
One brief, four results: run 1 of 3 for each, Claude Opus 5.5. Compare any two skills →
How it works
- 5 fixed briefs: a SaaS landing page, an analytics dashboard, a mobile app screen, a sign-up form, and SkillArena's own home page. Each is sent word for word.
- One model, default settings: every page comes from Claude Opus 5.5. The only thing that changes between rows is which
SKILL.md goes into the system prompt. One row has no skill at all.
- 3 runs per skill and brief, all shown, so you see the variation and not a cherry-picked best.
- Pinned and reproducible: each skill is fetched at a fixed commit, and its page has install commands for that exact version. Skills that need to run bundled scripts are skipped and listed in
skipped.md.
- No ranking: you look and decide. All runs, token counts and costs are in
results.json.
What the results show so far
From the published set of 117 skills plus the control group, 1,770 pages (November 2026):
- Skills change the look much more than the words. On the landing page brief, 317 of 354 pages open with “Stop asking…”, a phrase the brief never uses. Only 2 skills avoided it in all three runs.
- More instructions don't always mean more output. With no skill, the model averaged 12.6k output tokens per page; 21 of the 117 skills came in under that. It also depends on the brief: on the mobile app screen 114 of 117 skills wrote more than no skill, on the dashboard 82 wrote less.
- One screenshot is not enough. The same skill on the same brief went from 17k to 44k output tokens across its three runs.
Want a skill tested? Open an issue with a link to its SKILL.md. See CONTRIBUTING.md.
What's in this repository
This repository holds everything behind the published results:
| Folder | What |
|---|
pipeline/ | Fetch skills at a pinned commit → generate pages → screenshot → scan → manifest.json |
config/ | The skill list (skills.yaml), the briefs, and the run settings (model, runs per pair, budget) |
output/ | One meta.json per run (status, model, tokens, cost, screenshot info). Screenshots and pages are on the asset domain |
web/ | The site: Next.js static export, no client-side framework runtime |
contract/ | zod schemas for the public API and the build data (manifest, display settings) |
functions/ | Reference implementation of the public API as Cloudflare Pages Functions |
Quick start: build the site from the published results
Needs Node 22+ and Yarn 4 (corepack enable).
yarn install
cp .env.example .env # ASSET_BASE points at the published screenshots; no keys needed
yarn build # static site in web/out
yarn dev # or run it on http://localhost:4321
Nothing here calls a model: the site is built from web/data/manifest.json. Without NEXT_PUBLIC_API_URL the analytics beacon is off and the waitlist / try forms are hidden.
To also have the run files locally (for the scans, or to rebuild manifest.json without ASSET_BASE): yarn fetch-assets.
Requirements for generating results
- Playwright Chromium (
npx playwright install chromium). playwright is pinned to 1.61.1.
- Either an Anthropic API key in
.env (--backend api, the default) or a logged-in Claude Code CLI (--backend cli).
- The
gh CLI (logged in) for yarn discover, which ranks candidate skills by repo stars.
Running the pipeline
yarn pipeline -- --dry-run # call count and cost estimate, then stop
yarn pipeline -- --skill anthropic-frontend-design --brief saas-landing --runs 1
yarn pipeline -- --mock # placeholder HTML, no model calls ($0)
| Flag | Meaning |
|---|
--skill a,b | Only these skills. baseline is the control group. |
--brief x,y | Only these briefs. |
--runs n | Runs per pair (default 3, config/run.config.ts). |
--force | Regenerate / re-screenshot even when a result exists. |
--yes | Skip the cost confirmation. |
--dry-run | Only print the plan and cost estimate. |
--backend api|cli | Anthropic API (needs ANTHROPIC_API_KEY) or the local claude CLI login. |
Steps on their own: yarn fetch-skills, yarn generate, yarn screenshot, yarn manifest [-- --ready-only], yarn og.
yarn github-meta records each skill repo's owner (login and name), avatar and star count in web/data/github.json, and downloads the avatars into web/public/avatars/ so the site serves them itself (visitors never load images from GitHub). yarn manifest merges them into each skill; star counts carry the date they were fetched. Uses GITHUB_TOKEN when set.
- Idempotent: runs with status
ok or invalid_html are skipped; failed runs (API errors) are retried.
- Cost safety:
generate prints the call count and estimated cost, asks before more than one call, refuses estimates over budgetUsd ($30) and stops when spend reaches it.
--ready-only leaves out skills that still have unfinished runs, so the site can be rebuilt while a batch is generating. A truncated run (invalid_html) counts as finished and is shown as “Generation failed”.
以上为 README 的开头部分。其余内容请阅读原文。 查看完整 README ↗
