Mount any phone above your work: a phone repair, a cutting board, a flat-pack bookshelf. Second Hands finds the real guide on the web, turns it into steps a camera can check, watches every step, and speaks up only when it matters:
| 🔍 Finds the guide | 👁 Watches your hands | 🗣 Speaks up when it matters |
|---|
| Tavily pulls real guides at runtime. Nemotron Super VL turns them into steps a camera can verify. | Nemotron Super VL checks each frame against the step you're on and boxes what it's looking at. | Lightning decides whether to interrupt. Ultra double-checks the evidence before it ever says "Stop." |
📸 See it work
🧩 The problem
One wrong move in a hands-on job costs real money, or a finger, and nobody is watching:
- 🔋 Unplugging a phone's display while the battery is connected can short the board.
- 🔩 A long screw in a short hole can crack the board underneath.
- 🔪 Flat fingers in front of a knife is how most kitchen cuts happen.
- 🪚 A bookshelf without its anti-tip strap can fall on a child.
Manuals and videos can't see what you are doing. An expert next to you can, but experts are expensive, and a repair shop owner can't watch every apprentice all day. Second Hands is that expert, sitting above the bench.
Built for phone repair apprentices first (shaped with input from a working phone repair shop), where one slip costs a customer's phone. The same loop works for any hands-on task with a guide:
| 🔌 Electronics | 🍳 Cooking | 🪚 Furniture | 🏠 Home fixes | 🚲 Bikes | ✂️ Crafts |
|---|
| battery first, no metal on connectors | claw grip, cut away from your hand | panels the right way round, anti-tip strap | tap cartridges, drywall patches | flats, brakes, chains | soldering, sewing |
Tasks where a missed warning could seriously hurt someone (mains electrical, gas, vehicle brakes, firearms) are refused.
✨ What's in the box
📱 The phone app (installable, no app store)
- Full-screen HUD over the live camera: segmented step bar, scan line, and a focus box drawn from the vision model's output
- Pair in a second: scan the laptop's QR (native
BarcodeDetector, jsQR fallback) or type the 8-character code
- Camera check before starting (light, focus) and a flashlight toggle (Android)
- Hands-free voice commands: "skip", "back", "repeat", "check", "I'm right", "pause". It never reacts to its own voice.
- Manual controls: Back · Check now · Skip · Pause, so the person stays in charge
- Spoken warnings with a chime before every Stop; the screen stays awake
💻 The bench dashboard
- Procedure checklist, live view about 1 s behind the camera (a small preview stream that never touches a model), evidence panel (the frame next to the manual step), console-style event log
- Agent trace: every Token Factory call with stage, model, latency, tokens and cost
- Session report: steps done, mistakes caught, false alarms, duration, cost, and a timeline with the frame behind every warning
🧪 Built to be judged
/try: one click, no camera needed. It plays a clearly labelled recorded run while the real-footage sample clip is being recorded, then runs the live pipeline on that clip (with the replay as a fallback if the daily demo budget is used up)- Per-IP limits, a daily budget cap and a 5-minute cap per live session keep the public demo up through judging
⚙️ How it works
| Step | What happens |
|---|
| 1 | Start · /start | Pick a category and type the job ("Redmi Note 8 battery replacement", "Dice an onion safely"). Tavily finds real guides; Nemotron builds the checklist with what "done" looks like from above, the costly mistakes to watch for, and where to put the camera. |
| 2 | Pair · /cam | Scan the QR with any phone (or install the app). Camera check, then it starts watching. |
| 3 | Watch | The phone sends a frame whenever the scene changes. The dashboard shows the live view, checklist, events and every model call. |
| 4 | Catch | The same mistake in 2 of the last 3 frames → Lightning decides if it's worth interrupting → on high-severity steps Ultra verifies → the phone chimes: "Stop." |
| 5 | Ask | When unsure, it asks instead: "Is that the long screw? Hold it closer." I'm right silences it for that step. |
| 6 | Report · /report | Steps done, mistakes caught, false alarms, duration, cost, and the evidence timeline. |
The catch loop, from frame to spoken warning:
flowchart LR
F["📷 Frame<br/>(on change)"] --> VL["👁 Super VL<br/>observe"]
VL --> C{"2 of last 3<br/>frames agree?"}
C -- no --> F
C -- yes --> N["⚡ Lightning<br/>warn · ask · silent"]
N -- silent --> F
N -- "high severity" --> U["⚖️ Ultra<br/>verify"]
N -- "low / medium" --> A["🔔 Ask or warn"]
U -- confirmed --> S["🛑 STOP"]
U -- "not sure" --> A
📱 Works on any device
Everything runs in the browser over the internet, so the camera and the dashboard don't need to share a Wi-Fi network, and nothing is installed from an app store.
| Camera | Dashboard | |
|---|
| Android phone (Chrome) | Laptop / PC, or another phone | ✅ Tested on a Poco F1 and a Moto G85 |
| iPhone (Safari) | Laptop / PC, or another phone | ✅ Tested in WebKit with iPhone 15 emulation: QR pairing, frames, live view |
| Laptop webcam | The same laptop | ✅ Point the webcam at your hands |
| No camera | Any browser | ✅ /try sample run |
On iPhone, add it with Share → Add to Home Screen. Safari doesn't let web apps control the flashlight, and voice commands depend on the iOS version; the on-screen buttons always work.
여기까지가 README 앞부분입니다. 나머지는 원문에서 읽을 수 있습니다. README 전문 보기 ↗
