Glyph Arena
Find out which AI recipe is best for your own work, on your own tasks.
Glyph Arena is the lab of the Glyph family. It replays the
tasks you actually gave your AI tools, runs them on two recipes at a time (a recipe is
a model plus its prompt style, vault context and tools), and writes a routing policy
that Glyph Studio follows: which recipe to
use for which kind of task, with a sentence on why.
- Ghost replays of your own past prompts, sorted into leagues by domain, kind of
task (drafting, decisions, research, planning, video scripts, coding) and weight class.
- Cheapest honest verdict first: a mechanical checker (a number that must reconcile,
valid JSON, code that passes tests, a word limit), then agreement between the answers,
then a judge for questions with a known answer, and only then a two-second blind
pick by you. The unblinding score shows how often your blind pick matched the
current route.
- Equivalent, use the cheaper: when two recipes are even, the cheaper one gets the route.
- Home Only league: health and money domains run on local models only (Ollama on
your own machines), refused before any call, never sent.
- Adverse-event register: flag an answer that made something up or gave unsafe
advice; one severe event, or two major ones in 30 days, suspends that model in that
domain for 30 days.
- Monthly spend cap checked before every cloud call; weekly sentinels catch a
model that silently got worse; scouting lists new local models and the web
scout's picks as challengers.
Everything is plain files in your vault, under build/_meta/arena/: recipes.json,
duels.jsonl, picks.jsonl, adverse.jsonl, spend.jsonl, and the result,
policy.json with its readable twin policy.md.
How it fits together
Arena's logic lives in glyph-engine
(src/arena), next to the model providers it uses, so there is one implementation.
This app is the interface: a Tauri shell that runs the engine's arena-engine sidecar,
a separate program (GPL-3.0, source in glyph-engine). Studio's Auto router reads
policy.json from the same vault. Nothing here needs Studio installed.
Build
Needs Rust, Node, Bun and a checkout of glyph-engine.
npm install
ENGINE_DIR=../glyph-engine bash scripts/prepare-sidecar.sh # or ENGINE_REF=main to clone it
npx tauri build
Tests: npm test drives the bundled sidecar on a made-up vault (Home Only refusals, scoring, the policy file); cargo test in src-tauri covers the shell. The engine's own suite is in glyph-engine (bun test src/arena).
For UI work in a browser: node scripts/demo-vault.mjs /tmp/arena-demo, then
ARENA_VAULT=/tmp/arena-demo bun scripts/dev-bridge.ts and npm run dev. The demo vault
is entirely made up.
License
MIT for this app. The bundled arena-engine is built from glyph-engine (GPL-3.0).
