CV Playground
English | 日本語
A web page for trying out and comparing computer vision models right in the browser.
On the same screen you can choose "run in the browser" (WebGPU / WASM on your device) or "run on the server", and see inference time, fps, and a per-stage breakdown. That makes it easy to check how far a model actually goes on a phone or a laptop.
Open it at https://yoshiri.github.io/cv-playground/ (no server; browser-only models; works on phones).
The UI is in English or Japanese: it follows your browser's language, and the button at the top switches it (remembered per browser; ?lang=en / ?lang=ja in the URL also works). The developer docs under docs/ are in Japanese.
Demo
| | |
|---|
| Human pose + tracking (YOLO26n-pose + ByteTrack, with IDs and trails) | Object detection (YOLO26n) | Hands and eyes (PINTO's DEIMv2 + OCEC eye open/closed) |
| Semantic segmentation (EoMT DINOv3, ADE20K) | Panoptic segmentation (EoMT DINOv3, COCO; each chair and desk separately) | Promptable segmentation (SAM 2.1, clicked person) |
| Segment everything (EdgeTAM, from a 144-point grid) | Depth estimation (Depth Anything 3, also estimates the field of view) | Text-prompted object detection (Grounding DINO, "orange, lemon") |
| Zero-shot classification (SigLIP2) | Background removal (BiRefNet lite) | Faces + expressions (YuNet + HSEmotion; expression, valence V and arousal A) |
| Template matching (XFeat + RANSAC; the shelf cut from an earlier frame, found again with people in front) | Go / stop (detection + depth + scene judgment; stops with the reason) | |
The demo frames come from intel-iot-devkit/sample-videos (CC BY 4.0).
Features
Tasks (tabs, in four categories)
| Category | Tab | What it does |
|---|
| Detection & tracking | Object detection | 80 COCO classes (YOLO26, RF-DETR), with tracking on video and camera |
| Human pose | 17 keypoints, with tracking |
| Face | YuNet with 5 landmarks; cropped faces classified by HSEmotion into 8 expressions plus valence and arousal |
| Hands & eyes (PINTO) | PINTO's ultra-light models: body, head, face, eyes, hands, plus eye open/closed, pointing and waving |
| Gestures (pose + PINTO) | Both run on each frame: hand raised, face direction, eyes closed, pointing, per person |
| Text-prompted detection | Grounding DINO with English nouns |
| Template matching | Drag a box (or pick an image) and find it with XFeat keypoint matches and a homography; matches can be drawn as lines |
| Video stabilization | Frame-to-frame motion from XFeat, re-rendered along a smoothed (or locked) camera path; jitter before and after as numbers and a graph |
| Segmentation | Promptable (SAM) | SAM 2.1 / SAM 3 / EdgeTAM: click points, or segment everything from a grid |
| Semantic / Panoptic | SegFormer / EoMT (ADE20K), DETR / EoMT (COCO panoptic, each object separately) |
| Background removal | BiRefNet |
| Depth & 3D | Depth estimation | Depth Anything V2 / 3 |
| Approach / recede | Detection + depth: a small Kalman filter per track classifies approaching or receding, estimates time to contact and counts per class |
| 3D pose | Pose + depth: a rotating skeleton with depth |
| Go / stop | Detection, depth and a scene judgment on the same frame; stops when something is ahead, approaching, too close, or the scene is judged blocked, and shows why |
| Vision & language | Zero-shot classification | SigLIP2; also yes/no judgments (see below) |
| Captioning & VQA | SmolVLM / Qwen3-VL, or any image-capable model on Ollama (server) |
Layers on top of the tasks
| Layer | What it does |
|---|
| Judgments | Yes/no phrases such as Passable | Blocked or Safe | Dangerous in the zero-shot tab are overlaid on the image; on video they are smoothed over time (with hysteresis) and changes are logged. Go / stop combines them with detection and depth |
| Applications | Add-ons for any tab that returns boxes: counting per class (current count and, with tracking, a cumulative count of IDs), and line crossing (draw a line on the image; counts how many tracks cross it, per direction and class, and streams each crossing as an event). ?apps=count,line turns them on from the start |
| Interaction | Streams results out in a fixed frame format: to other tabs (BroadcastChannel) or over WebSocket (the server relays it at /ws; see tools/ws_receiver.py), so other tools can use the page as a browser-based vision sensor. web/receiver.html shows the incoming frames, and a Live2D-style character that follows the largest person serves as a visual check. ?interact=broadcast,websocket,puppet turns them on from the start |
Everything else
- Input: sample images and videos (one click), images, video files, and live camera (back, front, or any camera on the device). Video and camera run continuously with results overlaid, showing fps and a breakdown (grab, preprocess, model, postprocess). The screen stays awake during continuous runs
- Tracking: ByteTrack / BoT-SORT / BoT-SORT + ReID (ported from Ultralytics; verified to give identical results on the same detection sequence)
- Runtime options: by default, generic ONNX models run with fp16 (when the model has an fp16 file) and WebGPU graph capture, which is dropped automatically for models that cannot use it. You can switch to fp32, no graph capture, or CPU (WASM), and pick the input long side (320–960) for models with dynamic input
- Compare: the same model in the browser and on the server (generic ONNX models build pre/post-processing from shared blocks, so both sides follow the same steps). Timings are kept in the run history
- Benchmark: on a fixed sample image, 3 warm-up runs followed by 5 / 20 / 50 / 100 measured runs, recording median, p90, p95, and fps.
?profile=1 also gives per-operator GPU time for generic ONNX models (see docs/NOTES.md)
- Use local ONNX files: if you already have a published model's file, pick it and use it without downloading (only files whose SHA-256 matches exactly; generic ONNX models only)
- Export: the displayed image (with an optional caption strip; on phones, the share sheet lets you save to Photos), result data (JSON), and run records (CSV / JSON / Markdown table, in the UI language). Records include device, browser, and GPU info, so CSVs from different devices can be concatenated and compared as is
Data, network usage, and licenses
- Models that run in the browser are downloaded to your device on first use (from a few MB up to about 1.4 GB; the size is shown next to each model name). Watch your data usage on mobile networks. Models over 100 MB ask for confirmation before downloading, and later runs load from the browser cache. "Clear downloaded models" at the bottom of the page removes them. The browser cache is per site (origin) and may be evicted when the device runs low on storage. With the server, model files are fetched through the server, which keeps a copy in
models/mirror/, so other devices on the same tailnet/LAN, or a device whose cache was cleared, load them from the server instead of the internet (?mirror=0 turns this off). The result panel shows where the model was loaded from (browser cache, server, or download)
- Images, videos, and camera frames never leave your device as long as you run in the browser (the only network traffic is fetching models and libraries). Only when you choose "run on the server" is the image sent to the server
- Each model has its own license, and some do not allow commercial use (e.g., YOLO26 is AGPL-3.0; Depth Anything V2 Large and SegFormer are non-commercial). The license is shown in each model's description on screen
- Supported browsers: Chrome / Edge / Safari (iOS 26 or later) with WebGPU are recommended. Without WebGPU it falls back to WASM, which is slower
Usage
Three ways to serve it (all from the same code in web/)
| Form | What it is | Differences |
|---|
| Server | server.py serves web/ plus the API | Server-side models (on a Mac, etc.) are also available. COOP/COEP headers enable multi-threaded WASM |
| Static | GitHub Pages or any static host serves web/ as is | The API is unreachable, so only browser models. Multi-threaded WASM is enabled by the bundled coi-serviceworker |
| Single file | dist/cv-playground.html, built by python3 build.py | Just open the file (works from file://). Browser models only; single-threaded WASM |
The page detects at runtime whether a server is present (whether api/models responds).
Running the server
uv venv --python 3.12 .venv && uv pip install --python .venv/bin/python -r requirements.txt
cp .env.example .env # per-environment settings (optional; .env is not committed)
scripts/serve.sh # http://127.0.0.1:8010 (--bg to run in the background)
- Server-side models are downloaded from Hugging Face on first use (several GB in total). Up to 3 are kept in memory; beyond that the least recently used is unloaded
- To open it from another device (e.g., a phone), HTTPS is required (for WebGPU and the camera). With Tailscale, for example:
tailscale serve --bg --https=8443 http://127.0.0.1:8010
- On Apple Silicon it uses MPS / CoreML; elsewhere, CPU (CUDA is not supported or tested)
- To use YOLO26 with dynamic input size, export it with
tools/export_yolo26_dynamic.py (the weights are AGPL, so they are not in the repo and only the server serves them)
Environment variables (.env)
| Variable | Meaning | Default |
|---|
CVPG_PORT | Port to listen on | 8010 |
CVPG_MAX_LOADED | Number of server models kept loaded at once | 3 |
CVPG_OLLAMA_MODEL | Model used by the "Ollama VLM" entry | qwen3-vl:8b |
OLLAMA_HOST_URL | Ollama URL | http://127.0.0.1:11434 |
CVPG_GPU_LOCK | If this file exists, the page shows "GPU is in use by another process" (optional) | none |
Forking and extending
In most cases, adding one entry to web/models.json is enough (for plain ONNX models, even pre/post-processing is composed in JSON).
Layout
| File | Role |
|---|
web/models.json | Definitions of tabs (tasks) and models, shared by browser and server |
web/app.js | UI, continuous video/camera runs, wiring for tracking and cascades, run history, benchmark |
web/renderers.js | How each result kind is shown (boxes, masks, depth, segment maps, labels, text) |
web/judge.js | Judgments: yes/no from zero-shot scores, smoothing and hysteresis over time, and the go / stop conditions |
web/i18n.js, web/i18n_en.js | UI language. Japanese is the source text; English comes from a dictionary (exact strings, patterns with numbers, and phrases) applied to the page as it changes, plus *_en fields in models.json |
web/apps.js | Applications added on top of a tab's results (stateful aggregation across frames and its overlay, e.g. counting) |
web/interact.js | Interaction layer: the fixed frame format, outputs (BroadcastChannel, WebSocket) and the Live2D-style character |
web/receiver.html | A receiver page that shows the streamed frames (debugging) |
web/export.js | Image saving, result data, run-record export (CSV / JSON / Markdown), device info, statistics |
web/worker.js | Browser-side adapters (Web Workers; separate workers for onnxruntime-web and transformers.js) |
web/onnx_generic.js | Browser-side generic ONNX (pre/post-processing blocks) |
web/tracker.js | ByteTrack / BoT-SORT (+ ReID) |
adapters.py | Server-side adapters (the generic ONNX blocks mirror onnx_generic.js) |
server.py | FastAPI. Serves web/, /api/run, /api/status, /api/unload, /local-models/, /mirror/ (a disk cache of model files and sample videos) and /ws (the WebSocket relay for Interaction) |
build.py | Builds the single-file version (a stale build is caught by ) |
License
The code is under the MIT License (LICENSE). Model weights are generally not bundled; they are fetched from each distributor at runtime. Check each model's license with its distributor (e.g., YOLO26 is AGPL-3.0 from Ultralytics).
The exception is web/pinto/, which bundles models from PINTO_model_zoo (MIT) with attribution. Sample images (street, soccer, person, face, airport) are loaded at runtime from Xenova/transformers.js-docs on Hugging Face, and sample videos (corridor, parking lot, walking faces, store aisle, classroom) from intel-iot-devkit/sample-videos (CC BY 4.0, credited on the page); neither is stored in the repo. The README demo images are frames from intel-iot-devkit/sample-videos (CC BY 4.0) with results overlaid.
