VisionGuard
Website & screenshots: https://mupf-dev.github.io/vision-guard/
Self-hosted, Docker-based video analytics for network cameras. Camera streams are not recorded continuously but
analysed live by AI: objects, people, animals and vehicles are detected, tracked and – optionally – identified (faces,
licence plates, pets). VisionGuard also remembers where things are ("Where is my remote?"). Rules decide what happens:
snapshot, video clip from the RAM ring buffer, push notification to your phone, MQTT message or webhook.
The user interface, notifications, reports and AI-agent answers are available in English (default) and
German; more languages can be added with plain translation files (see Languages).
Features
- Live tile view (1–16 cameras) with bounding-box overlays, zones and a live event feed
- Object detection with YOLO11 on an NVIDIA GPU, batched across all cameras, ByteTrack tracking
- Identification: faces (InsightFace), licence plates (fast-alpr), pets (DINOv2 embeddings). Unknown subjects
automatically get an ID ("Person #12") and can be named or merged later. Faces are collected per track and rated
by quality (size, head pose, sharpness); for cameras above 1920 px they are cropped from the full resolution.
Multi-select to merge/delete with history and undo; a nightly run bundles fragments, curates the image gallery
and adapts per-person thresholds (default: dry run with preview only, applied after approval in Settings → Nightly
run)
- Rule engine: camera, zone, object class, confidence, identity (known/unknown/specific), trigger
(enter/leave/dwell/posture/gesture/time/sensor), schedule, cooldown → snapshot, clip, MQTT, webhook, push, PTZ, TV
- Virtual sensors and supplies: image regions as switches (light on/off by brightness, blinking, reference
images, counting items, fill level, asking the vision model: yes/no, count up to 3 or "empty") – usable as rule
trigger and condition, via MQTT and MCP; low supplies are added to the shopping list automatically
- Items (inventory): open vocabulary (YOLOE, 50 everyday objects, extendable), location ("probably on the
table"), changes since yesterday, your own specific items ("my key ring"), search in German
- Notifications with push to ntfy or Pushover (with image and answer links), rules with conditions (item
missing, weather, sensor), emergencies repeated until acknowledged
- Please confirm: for uncertain detections VisionGuard asks (UI, push, MCP) and learns from the answer
- Posture, falls and gestures (pose model): "lying on the floor" as a trigger, hand signals as switches
- Pets: pet diary (eating, drinking, litter box per zone)
- Presence: who is where right now, across cameras even without a face (re-identification, kept in memory only)
- Daily report (template-based, optionally written by a language model via an OpenAI-compatible endpoint) via
push, MQTT, webhook or e-mail
- Ask the camera: questions about the current image to a local vision model (Moondream, service
vlm, optional)
- Camera health: covered, blurred, too dark, frozen or moved – with notification
- Pan/tilt cameras (PTZ via ONVIF): control pad in the live view, save positions and the home position right
there, scheduled patrols, following people/animals, rule action "pan camera"; zones, items and sensors apply to
the home position and pause while the camera looks elsewhere
- Live view on your TV (DLNA): rule action "show on TV" (e.g. person at the door → camera full screen for 60 s,
then back to the programme), test button in the settings, MCP
show_on_tv; tested with Samsung (Tizen). The TV
must be able to reach VisionGuard on the home network (under WSL2: port forwarding, see AGENTS.md)
- Events with filters, snapshots, clips and statistics
- Administration: cameras (with connection test), zone editor, rule editor, settings, system status/GPU
- MQTT to the outside (e.g. Home Assistant): all events, counters per camera and object class, sensors;
read-only from outside (separate listener, see below)
- Time machine: per camera an optional image memory (image embeddings + 256 px preview, no video); searching in
your own words ("red box in the hallway") returns time ranges with a filmstrip – also via MCP
Languages
- The web interface language is chosen per browser (globe icon in the header); it defaults to the browser language
and falls back to English.
- The system language (Settings → Language) is used for everything created without a browser: notifications and
push messages, the daily report, "please confirm" questions, MQTT texts and MCP answers.
- Adding a language needs no code changes beyond registering it: copy
frontend/src/i18n/locales/en,
backend/app/locales/en and mcp/locales/en.json, translate, and add the language to
frontend/src/i18n/locales/index.ts. Missing strings fall back to English. Details in AGENTS.md
(German).
Architecture
Cameras (RTSP) ──▶ go2rtc ──WebRTC/MSE──▶ Browser (React UI, nginx)
│ ▲ REST + WebSocket
│ RTSP (sub stream) │
▼ │
worker (GPU) ──MQTT──▶ mosquitto ◀──▶ backend (FastAPI) ──▶ PostgreSQL (pgvector)
YOLO11, ByteTrack, ▲ rules, events, media volume
face/plate/pet, │ GPU lease identities, items, (snapshots/clips)
YOLOE items, pose, │ notifications, auth
RAM ring buffer for clips │ ▲
vlm (vision model) │ REST
mcp (AI agents, /mcp)
以上为 README 的开头部分。其余内容请阅读原文。 查看完整 README ↗
