vlmkit
Deterministic verification toolkit for frontend work: markup gates
(broken-page scan, copy fidelity, responsive boundaries, keyboard
operability, scripted flows), visual regression over time, design
audits, and repair tools. Built for humans and coding agents alike —
every failing gate prints a machine-parsable fix list. Everything is
key-free unless marked [key].
Requires Node 24+.
Quickstart — visual analysis with zero context
npm install -D @mizchi/vlmkit
npx playwright install chromium # once
# 1. find concrete visual defects on the page you're working on
# (no reference, no config, no API key — 3 viewports at once)
npx vlmkit check integrity http://localhost:3000/
# -> verdict: DEFECTS … [page-overflow-x] @768: … main > p (right edge 924px)
# [text-collision] @1280: "Total: €1,240" overlaps "Refunds: €80"
# fix what it names, re-run until: verdict: CLEAN
# long-polling page: choose an earlier navigation milestone
npx vlmkit check integrity http://localhost:3000/ --wait-until domcontentloaded
# 2. track what your edits change visually (1st run = baseline, then diffs)
npx vlmkit snapshot http://localhost:3000/ --output .vlmkit/snapshots
# -> desktop 13.11% … mobile 6.57% + diff heatmaps next to the report
# intended? npx vlmkit snapshot approve --output .vlmkit/snapshots
From there: check copy --manifest (exact copy), check breakpoints --sweep (responsive boundaries), check interactions (keyboard
operability), verify markup --target design.png (match a design) —
the full task-routing table, done-condition recipes, and agent
integration (MCP server + markup-assist skill) live in
docs/markup-assist.md.
Demos
mizchi.github.io/vlmkit/demos/ — every feature on a page with a
known defect: the command, its whole output, and screenshots from the same run. Most pages are this
repository's own test fixtures, so each command works from a clone after pnpm build.
Six whole sites built by agents, each beside the log of how it was judged, are at
/sites/.
Setup
npm install -D @mizchi/vlmkit # or -g
npx playwright install chromium # once
vlmkit declares playwright as a peer and @playwright/test as an optional
peer, reusing the versions already selected by the project instead of
installing a second browser build.
- Coding agents (MCP) —
.mcp.json:
{ "mcpServers": { "vlmkit": { "command": "npx", "args": ["-y", "@mizchi/vlmkit", "mcp"] } } }
- Agent skill — install the automatic
vlmkit router once with APM or the
open skills CLI (commands below), then ask for the result in natural language.
- API keys — only for
[key] features (heal markup,
check copy --vlm, fix-loop): OPENROUTER_API_KEY /
GEMINI_API_KEY / ANTHROPIC_API_KEY.
- Snapshot / CI config —
vlmkit.config.json (routes, thresholds)
via vlmkit workflow init.
Full setup detail: docs/configuration.md.
Install agent skills
Recommended: install the vlmkit skill package once. You do not choose a
specialized skill; the agent classifies each frontend request, loads the
matching bundled workflow, runs its deterministic gates, fixes failures, and
reruns to green.
# Install or update APM (macOS / Linux)
curl -sSL https://aka.ms/apm-unix | sh
# Install vlmkit with APM
apm install mizchi/vlmkit
# Or install with the skills CLI (APM is not required)
npx skills add mizchi/vlmkit
Inside Claude Code the same package installs as a plugin — no shell, and
/plugin marketplace update picks up new workflows:
/plugin marketplace add mizchi/vlmkit
/plugin install vlmkit@vlmkit
The APM bootstrap above is the official recommended installer
and resolves the latest binary for the current platform. It avoids depending on
a stale package-manager formula.
Then ask naturally: “implement this mock,” “check this page's responsive and
keyboard behavior,” or “turn this story into stable Playwright tests.” There is
no skill-selection step in the normal workflow. On first use, the agent detects
the repository's package manager, reuses or adds @mizchi/vlmkit locally, and
installs Chromium only if a selected gate reports it missing.
All three installers expose one visible vlmkit skill. The 13 specialized
workflows are internal resources bundled under that entry, so the agent selects
them without adding 13 separate skills or copying the vlmkit source repository.
See the agent skill catalog for all 13
specialized skills, grouped by general verification, UI creation, test
generation, comparison and monitoring, and evaluation and hardening.
Explanation and figures (animations, D2 diagrams and slides)
live in mizchi/explainer.
When to use what
vlmkit --help prints this same map. Everything is key-free unless
marked [key].
| You want to… | Reach for |
|---|
| Check the page you just wrote/edited, no reference | check integrity (broken-page scan, 3 viewports) · check copy --manifest (copy present, visibly, verbatim) · check layout --contract · scan scroll / scan handlers |
| Verify behavior, not pixels | check breakpoints --sweep · check responsive (every width, shrunk to breakpoint + declaration) · check interactions · check scroll / check animation · verify flow --flow |
| Drive the page from a screenshot (computer use) | check grounding — action map in screenshot px, which element a click at each point actually reaches, --mark for a numbered overlay |
| Match a target design | verify markup --target (done verdict + fix list) · build page / build component · scan mock (normalize @2x exports) |
| Track changes over time (VRT) | snapshot (baseline → per-viewport diff → snapshot approve) · watch · diff-pr · baseline |
| Compare two versions | diff html|png|elements · migration compare · check equivalence --region |
| Audit design quality | check tokens|theme|palette · check design (is the page consistent with itself — one button style or six) · check a11y contrast|touch|focus · check perf · check drift · stress i18n|media |
| Vet an image asset before it enters a slot | check asset --slot --expect-transparent --against-bg --page-palette |
| Repair | heal selector (dead selector) · heal markup [key] (LLM auto-fix from a kickback) |
| Run gates over a whole site | batch --gate "check integrity" "routes/**/*.html" (parallel, --shard i/n, per-job timing) · gates run (same, from one reviewed config) |
| Audit what has been silenced | — every suppression with reason, owner, expiry; expired ones stop applying |
Task-routing recipes and done-condition sets:
docs/markup-assist.md.
With a key: TypeSafe Jev
Four tools ask TypeSafe Jev typed questions — a box, a yes/no (noul),
a choice — with TYPESAFEAI_API_KEY. The model says what is there and where; the
values come from the pixels and the page. It is never asked to name a measurement.
Jev is opt-in. Where a command works without it, --jev turns it on (diff png --jev,
inspect motion --jev). inspect locate, inspect claims and a describe click have no other
engine, so they need the key and nothing else.
| Feature | Command | Jev answers | vlmkit measures | Measured |
|---|
| Classify each diff region | diff png --jev | A choice per region: text, colour, position or presence | The regions themselves, from the pixel diff | Text 14/14, colour 18/18, presence 17/17; 1–6px shifts 11/16, every miss marked uncertain |
| Find an element from a description | inspect locate, click with describe | Is it shown (noul), a box, then a box again on a magnified crop | The crop, cut from the original pixels; the point, in click coordinates | 94% of targets hit; 0 of 85 decoys accepted |
| Check statements about a screen | inspect claims --forbid/--expect | One noul per statement, all in one request | Nothing: it advises and never gates | 54 of 55 planted problems caught; 0 false alarms on 43 clean pages |
| Is an animation visibly broken? | inspect motion --jev | Per frame: is each target shown, its box, a noul per forbidden statement | Frames on the held clock; jumps, snap-backs and easing fitted from the boxes; with --selector, geometry from the page | 7/7 broken caught, 7/7 intact passed (deterministic gates: 3/7); ~$0.002 a page. Without --jev, from --selector alone and no key: 6/7 and 7/7 (text left on text needs Jev) |
Asked to name an easing, Jev was at chance (7 of 21); fitted from its boxes, 21 of 21. Reports:
region kinds,
locate,
claims,
easing,
motion.
The workspace
flowchart BT
subgraph L0["layer 0"]
p7["judge"]
end
subgraph L1["layer 1"]
p4["core"]
end
subgraph L2["layer 2"]
p1["ai"]
p2["animation-eval"]
p3["capture"]
end
subgraph L3["layer 3"]
p5["generate"]
p6["heal"]
p8["markup"]
p10["plan"]
end
subgraph L4["layer 4"]
p9["mcp"]
end
subgraph L5["layer 5"]
p0["vlmkit (cli)"]
end
p0 --> p1
p0 --> p2
p0 --> p3
p0 --> p4
p0 --> p5
p0 --> p6
p0 --> p7
p0 --> p8
p0 --> p9
p0 --> p10
p1 --> p4
p1 --> p7
p2 --> p4
p2 --> p7
p3 --> p4
p4 --> p7
p5 --> p1
p6 --> p1
p6 --> p3
p6 --> p4
p8 --> p1
p8 --> p2
p8 --> p3
p8 --> p4
p8 --> p7
p9 --> p4
p9 --> p8
p10 --> p1
Eleven packages, layer by layer from judge (no dependencies) to the CLI; an arrow points from a
package to one it depends on. Generated from the manifests by scripts/workspace-map.mjs,
and a test fails when this block drifts from them.
Examples
| Example | What it shows |
|---|
| Feature demos | One page per feature: a page with a known defect, the command that finds it, its output and screenshots |
| Demo sites | Six sites built by agents with vlmkit's gates, each published beside the log of how it was judged |
| Component-focused VRT | check story on one component instead of a whole page, with a gallery to copy |
| Markup loop project | A minimal standalone project that reproduces the drop-in markup loop |
| Markup VRT eval | A dogfood harness that measures the Playwright workflow while a new screen is built |
| Gate plugin | A project that adds two gates of its own through vlmkit.config.json |
| Solitaire | Klondike in plain HTML/CSS/JS, the drag-and-drop and animation dogfood target |
| Intro page | The dependency-free landing page, verified with vlmkit's own loop |
Documentation
| Doc | Contents |
|---|
docs/introduce.md | What is vlmkit? — narrative introduction assuming zero context |
docs/markup-assist.md | Task-routing guide: which gate for which job, done-condition recipes, anti-gaming rules |
docs/cli-reference.md | Complete command reference: all groups, examples, workflow/API/HTTP, architecture, project structure |
docs/configuration.md | Install, MCP/skill setup, environment variables, snapshot/CI config, agent skills catalog |
packages/vlmkit-mcp/README.md | MCP tool table |
docs/knowledge.md · docs/reports/ | Accumulated evaluation findings and dated experiment reports |
CHANGELOG.md | Release notes, including the 0.4.x → current command mapping |
License
MIT
