Throw Hands
A browser fighting game where your body is the controller. Your webcam tracks your pose, your 3D superhero fighter copies your movements, and your real punches, blocks, and special-move gestures fight a bot, a friend next to you, or friends on other devices.
Play: https://throwhands.tech (same server, backup address: https://144-202-71-213.sslip.io)
Built at HackTX 2026 (October 24 to 25, 2026). Built with Claude Code (AI coding tool, disclosed per the event rules).
What you can do
- Move with your body: step or lean toward the camera to advance, back to retreat, sideways to circle (a short tutorial teaches it after calibration). Your fighter walks where you step.
- Practice vs a bot in a UFC-style octagon: best of 3, 60-second rounds, HP KOs, big hits slam fighters into the cage, rematch.
- 2 players, 1 camera: two people side by side each control a fighter.
- Online 1v1: create a room, share the link or 4-letter code, fight a friend on any device. Spectators can watch and send cheers.
- Brawl: up to 8 fighters in a larger octagon, bots fill empty slots, knockback that grows with damage taken, 3-second respawns, live KO feed.
- Winner stays on: up to 8 friends, two fight at a time, the winner stays on.
- Guard: both forearms up in front of the face blocks 90% of damage, but a GUARD meter drains while you hold it and when hits land on it; let it empty and your guard breaks (dizzy for 2 s).
- Super moves: uppercut (fist driven up), Iron Wall (forearms crossed in an X above the face, elbows up: 2.5 s unbreakable guard that counters the first hit), energy blast (both palms pushed forward), when your meter is full.
- Grappling (beta): on by default; add
?grapple=0 to turn it off. Reach with both arms and pull in to grab into a clinch, then lift a knee for knees (8 damage); from the clinch, turn and bend for the Shoulder Throw (25 damage, needs a full super meter). Online rooms play without grappling.
- Fighter Forge: describe a fighter; Meta's Muse Spark writes the name, story, and moves, Muse Image paints the card, and your suit takes the card's colors.
- Commentary: an announcer calls the big moments instantly (pre-rendered ElevenLabs calls) over a real crowd recording; Gemini writes live lines about the fight (shown as captions).
- Rivalry card after online matches, Elo leaderboard (Tiger Data), store (mock purchases, "Demo: no charge"), and an opt-in motion data program with an explicit consent screen (labeled move sequences plus continuous 15 fps body-point streams, capped at 10 minutes per match; coordinates only, never video; stored in compressed Tiger hypertables). A separate research mode (
?research=1, consent every session) records the camera to a local file only and sends labeled landmarks for gesture tuning.
No video ever leaves your device: pose tracking runs in your browser. Online play sends only joint positions and game actions.
How it works
- Pose: MediaPipe Pose Landmarker (lite) in a Web Worker, GPU delegate with CPU fallback. Camera frames are read straight from the camera track (
MediaStreamTrackProcessor) where supported, so pose rate does not depend on render rate.
- Retargeting: our own solver turns 15 world landmarks into bone rotations for a VRM humanoid (
web/src/game/avatar.ts), mirror-correct, with a One Euro filter for low lag.
- Gestures: rule-based (
shared/gestures.ts): strikes are detected as elbow-angle / wrist-height / sideways-sweep peaks, so they work from 10 to 30 fps; guard and the Iron Wall X from measured arm geometry; a 3-second guard calibration sets body size. Tuned and scored against labeled video clips (see TESTING.md).
- Game: one deterministic simulation (
shared/sim.ts) for any number of fighters. It runs in the browser for local modes and on the server for online rooms, where the server decides every hit using the defender's latest block state.
- Server: Node +
ws on a Vultr VM in Dallas behind Caddy (automatic HTTPS). It relays rooms at 20 Hz, runs bots, and proxies Gemini, the Meta Model API, and Tiger Data so no key reaches the browser.
- Data: Tiger Data (PostgreSQL + TimescaleDB):
match_events and pose_samples hypertables; leaderboard_daily and motion_label_counts continuous aggregates with real-time aggregation and refresh policies.
Run it locally
npm install
npm run build # frontend to dist/, server to dist-server/
TIGER_DATABASE_URL=... GEMINI_API_KEY=... META_API_KEY=... npm run server # API + WebSocket on :8787
npm run dev # Vite dev server with /api and /ws proxied to :8787
Every integration fails soft: without keys or a database the game still runs (house fighters, template commentary, no leaderboard).
Tests: BASE_URL=https://throwhands.tech npx playwright test (see TESTING.md; video clips are not in the repo).
Deploy: deploy/deploy.sh (builds, rsyncs to the VM, restarts the service).
Prize tracks entered
- Most Entertaining (Jester's Court)
- Meta: Bringing People Closer Together with AI
- MLH: Best Use of Gemini API, Tiger Data, Vultr, .Tech domain
- Best Overall and Best Pitch (automatic)
What we wrote vs. third-party code (Meta rules: original contribution clearly identified)
Everything in web/src, shared/, server/, tests/, scripts/, and deploy/ was written during the hackathon (with Claude Code). That includes the pose-to-bone retargeting solver, gesture recognizer, game simulation, bots, networking, AI integrations, UI, synthesized sound effects, and tests. No code was vendored or copied into the repo; Kalidokit was not used.
Third-party libraries (installed from npm, unmodified):
This is the beginning of the README. Read the rest at the original. Read the full README ↗
