Gesture Gallery
Browse a photo gallery with hand gestures, right in the browser. Webcam optional: every gesture also works with keyboard and mouse.
Try it live → Desktop Chrome or Edge. Hand tracking runs on your device; the camera feed never leaves it.
Recorded on an earlier build.
Gestures
| Pose | Motion | What it does |
|---|
| ✋ Open palm | Move your hand | Cursor follows; the photo under it is outlined |
| 👆 Point | Dip the fingertip | Select the photo under the cursor, or press the button under it |
| 👌 OK sign (thumb touches index, other three fingers up) | Over a photo | Grab it. A curled-finger pinch 🤏 won't grab |
| 👌 then move back | Move your whole hand away from the screen | The photo grows; open your fingers to let go |
| ✊ Fist | Move up or down | Scroll the grid |
On a first visit, a short tour teaches these one at a time and moves on as you do each one. Replay it from ⓘ → Take the tour, or add ?tour to the URL.
Keyboard and mouse
| Gesture | Keyboard | Mouse |
|---|
| Move palm | Arrow keys | Hover |
| Point and tap | Space | Click |
| OK sign, move back | Enter (Esc closes) | Double-click |
| Fist scroll | Page Up / Down, Home / End | Scroll wheel |
With photos selected, Remove (or the Delete key) and Clear appear next to the title. Remove can be undone (Undo or Ctrl/Cmd+Z), and nothing is really deleted. The gallery is usable before hand tracking loads, and if the camera is blocked, missing or busy, a small card says which and how to fix it.
How it's built, and why
One index.html: vanilla HTML, CSS and JavaScript, no framework, no build step. MediaPipe Hands (pinned, from a CDN) finds 21 points on the hand in each camera frame; everything after that is this file.
Reading the hand
- Poses from finger geometry. Each fingertip is compared with its middle joint to tell an open palm, a pointing finger and a fist apart. A pose must hold for ~90 ms before it counts, so the cursor doesn't flicker between modes.
- Grabbing needs an OK sign, not a pinch. A hand closing into a fist passes through a pinch-like shape. Requiring the other three fingers to stay up, checked on the current frame rather than the smoothed one, stops accidental grabs.
- Depth without a depth camera. The hand's size in the image stands in for distance: as it shrinks, the photo grows.
- A tap is a fingertip dip, measured relative to hand size within a 220 ms window, with a cooldown, so slow drift doesn't select things.
- A steady cursor. A One-Euro filter keeps it still when the hand is still and responsive when it moves.
For everyone, not just webcam owners
- The grid is a multi-select listbox with a single Tab stop. Selections, removals and previews are announced to screen readers.
prefers-reduced-motion removes decorative animation but keeps hand-driven movement, because that is the interface.
- The camera is requested directly rather than through MediaPipe's helper, which shows a raw
alert() on every failure.
- The tour coaches from what the camera sees ("keep your other three fingers up", "now move your hand back").
Light on the machine. In Chromium at 4× CPU slowdown, the page's own work per frame (not counting the hand model) went from ~14 ms to ~2 ms when idle, and from ~22 ms to ~9 ms with a hand moving:
- A hidden loading spinner was forcing a repaint every frame. It now animates only while visible.
- Only the photo being enlarged gets its own compositing layer, not all 200.
- One hit-test per frame finds the photo under the cursor, instead of measuring every tile.
- The model runs once per new camera frame (~30/s), not once per screen refresh (~60/s).
- Switching tabs releases the camera (its light goes off); it reconnects quietly on return.
Details
- The sidebar and top-bar icons are a decorative frame, deliberately not buttons, so nothing looks clickable that isn't.
- Photos are free-licence images from Unsplash via Lorem Picsum, a fixed one per tile, over a generated gradient that doubles as the offline fallback.
- Tuning constants (pinch, tap, scroll, smoothing) sit at the top of the
<script>.
Run it locally
- Clone the repository.
- Open
index.html in Chrome or Edge. Nothing to install or build.
- Allow the camera and raise one hand.
If your browser blocks the camera on file://, serve the folder instead (python3 -m http.server, then visit http://localhost:8000).
Tests
58 end-to-end tests in tests/ run in Chromium on every pull request. Playwright is used only for testing; the site itself has no dependencies.
Hand tracking is tested without a webcam: the tests swap MediaPipe's model for a stand-in that feeds simulated hand landmarks (hand() in tests/helpers.js). The page's real gesture code therefore runs as it would with a hand in front of the camera. Chromium's fake webcam covers the camera states.
npm install
npx playwright install chromium # first time only
npm test
Requirements
- Desktop Chrome or Edge (other Chromium browsers generally work; mobile and Safari aren't supported)
- A webcam for gestures (optional)
License
MIT © 2026 Divyanshi Rajauria
Designed and built by Divyanshi Rajauria, with AI assistance.
Independent educational demo. Not affiliated with, endorsed by, or connected to Apple Inc. or iCloud.
