VisionCompanion

VisionCompanion (originally called SeeWalk) is a phone app that describes useful things along a walk. It uses the iPhone's rear camera to spot hazards, then gives short spoken alerts through headphones. The goal is to add information a cane can't provide—not to replace a cane or tell anyone when it's safe to cross.

Open the app · Watch the demo

Features

• Spoken street alerts: Hear about potholes, curbs, steps, doors, construction, crosswalks, and obstacles at head height. A tone indicates direction, and alerts are kept brief to avoid constant narration.
• Ask questions by voice: Ask what's ahead, what's blocking your path, what you're holding, or ask the app to read a sign. You can also tap What's ahead?
• English and French: Switch the app, voice commands, and spoken answers between English and French.
• Choose a voice: Pick from four voices and preview them in the app.
• Hazard map: Hear about nearby sidewalk and crossing reports from other walkers and Ottawa's open 311 data. The map is also available on screen for companions.
• Clear status when something goes wrong: The app reports connection problems, a blocked camera, or an unclear image rather than implying the way is clear.
• Works from the phone: Add it to the iPhone home screen to launch it full-screen in Safari.

Alerts are intended as extra information, not a safety guarantee. VisionCompanion does not identify people and does not give instructions to cross.

Try it

The hosted app is at visioncompanion.vercel.app. Camera and microphone access require HTTPS, so open it in Safari on an iPhone. Tap Start walk and allow camera and microphone access.

To add it to the home screen, use Share → Add to Home Screen in Safari.

Run locally

You'll need Python 3.10+ and Node.js 20+.

1. Set up the server and add your Gemini and ElevenLabs API keys to server/.env:

   cd server
   python3 -m venv .venv
   .venv/bin/pip install -r requirements.txt
   cp .env.example .env
   

2. In one terminal, start the API:

   cd server
   .venv/bin/uvicorn main:app --host 127.0.0.1 --port 8000
   

3. In another terminal, start the phone app:

   cd web
   npm install
   npm run dev
   

Open the Vite URL shown in the terminal. For a quick UI preview without API keys, create web/.env.local with VITE_MOCK_API=1. To try the camera on an iPhone during development, serve the app over HTTPS with a tunnel such as cloudflared.

The hazard map needs a Tiger Data DATABASE_URL. See server/.env.example for the available server settings.

Development

cd web
npm test
npm run build
npm run lint

cd server
.venv/bin/pytest

Project layout

server/   FastAPI service for camera analysis, voice questions, and hazard data
web/      React phone app, camera, audio, map, and English/French interface
samples/  Street photos used for evaluation
docs/     Product notes, diagrams, and demo materials