按最近已完成的 8 小时观测回合间的 GitHub 星标增量排序
58 项开源技能，覆盖完整的医学研究全生命周期。由医生构建，基于真实出版物测试。
AI 翻译
个人 GitHub 账号下公开仓库的所有者可以登录并验证所有权，自行管理介绍，无需管理员批准。
登录并生成验证文件，在 30 分钟内将其提交到此仓库的默认分支。其他分支或分叉仓库中的文件无效。然后在下方检查文件，即可无需管理员批准直接管理项目。请保留文件以维持管理权限。
如需登录，将前往登录页面。登录后请回到此处，再次点击所选的验证按钮。
在项目文件夹中运行 /nomorevibe verify，即可验证所有权并自行管理介绍。
一分钟完成验证A camera that tells you what it sees, out loud and offline. On-device scene and page description from a fine-tuned vision model, live OCR, 601-class detection and LiDAR depth. Built for VoiceOver and TalkBack. Free on iPhone and Android.
翻译准备中
取自仓库 README
正在加载完整 README。 以下为已保存的摘录。
👁️ A camera that tells you what it sees — out loud, offline, on your phone. Point it at something and it names it. Hold it up to a bill and it reads the bill. Press one button and it describes the whole room in a sentence you can act on. No account. No signal. No server. Nothing ever leaves the phone. Free on iPhone and Android — no ads, no subscription, nothing to buy inside. This began as the eyes of a robot that watched a chicken coop.查看完整 README ↗
Point it at something and it names it. Hold it up to a bill and it reads the bill. Press one button and it describes the whole room in a sentence you can act on. No account. No signal. No server. Nothing ever leaves the phone.
It is a native iPhone and Android app that runs object detection, text recognition and a a vision-language model on the phone itself, and speaks what they find out loud.
Free on iPhone and Android — no ads, no subscription, nothing to buy inside.
How we pick the AI model (living benchmark) →
Next: Live Room Mode, the plan for real-time room narration →
This is a no-holds-barred app for blind and low-vision people, and it keeps getting better with every update. When a better on-device model shows up, we test it on real photos, publish the results in the living benchmark, and ship the winner.
The smartest model your phone can run fast (coming). On first launch the app runs a short speed check and picks one of four levels: Sharp for the newest phones, Clear for recent ones, Balanced for most phones, and Quick for older and budget phones. Every phone works, and newer phones get more detail. You can switch levels any time in settings.
Accuracy first when it matters (coming). Medicine, money, mail and labels always get the most careful model your phone can run, even if it takes a little longer. Looking around a room stays quick.
Live Room Mode (in the works). Hold the phone up and it describes the room, remembers what it already told you, and only speaks up when something new shows up or something needs your attention. Read the plan.
Beyond 601 objects. The live detector knows 601 kinds of things today. The vision model can already name almost anything it sees, and the goal is for the live view to know thousands of things, all still on the phone.
▶ Watch on YouTube: live detection at a concert, describing a scene, answering a question out loud, reading mail, and Help Me Aim talking you into a photo. All real screen recordings, all offline.
🎙️ On the podcast: Blind Android Users, Episode 307: Meet the Developer of RealTime AI Cam
| 🌎 Translation | 🐶 Detection | 🏠 Home Screen | 📱 App Info | 📏 LiDAR Distance |
|---|---|---|---|---|
| Offline Spanish→English | 601 object classes | Clean, native UI | Lightweight install | Pro model depth sensing |
Designed, written and shipped by Matt Macosko (NiceDreamzApps). The detector, the base language model and the inference runtimes are upstream work, credited below; the app around them is this repo.
ContentView.swift, HomeView.swift) and the camera pipeline with thermal throttling (CameraViewModel.swift)YOLOv8Processor.swift), tracking (ObjectTracker.swift), LiDAR distance (LiDARManager.swift)LlamaVisionEngine.swift, OnDeviceVisionNarrator.swift), a check that says what is wrong with a bad shot before the model runs (FrameQualityGate.swift), and model lookup with a size- and hash-checked install (VisionModelStore.swift, make_manifest.py). The model is used untrained under the LFM Open License v1.0, converted and quantized by Nice Dreamz; see and This began as the eyes of a robot that watched a chicken coop.
The robot had to know what it was looking at while standing in a coop, where there is no wifi worth trusting and no reason to believe a server will answer. So the recognition had to run on the machine itself. That one constraint never went away, and it is why every part of this app still runs on your device and not on somebody else's computer.
The camera turned out to be more useful than the robot. It moved to a phone. Then blind and low-vision users found it, and told us what it was really for.
Chicken is still class 109 of the 601. So are Fox, Raccoon, Owl and Eagle.
The model: LiquidAI/LFM3-Omni-2B (Liquid AI, LFM Open License v1.0). Up to iOS 1.3.5 and Android 1.22 the app ran Vision Narrator 0.8B (divinetribe/Vision-Narrator-0.8B-4bit-mlx)
It speaks one to three plain sentences about what the camera is actually looking at — where you are, what's near you and on which side, people, anything moving, hazards. Point it at mail and it leads with what the page is and who sent it, then the number that matters, then the deadline, then what happens if it's ignored.
This runs on a small vision-language model that ships inside the app: starting with iOS 1.3.6 (in App Store review) and Android 1.23 (coming next) that is LFM3-Omni-2B from Liquid AI, run by llama.cpp, instead of calling somebody's server. That's most of why the download is large, and it's the whole point. It works in airplane mode, on a plane, in a basement, in a country where your data plan doesn't.
It never refuses a shot. Dark, blurry, cut off — it says so in a few words and then describes whatever it can make out anyway. It will not hand the picture back and tell you to take another one.
This app is used every day by blind and low-vision people. The accessibility work isn't a checkbox at the end, it is the product.
A spoken countdown before the photo. Tap once, it counts three, two, one out loud with a tap you can feel each second, then fires — so your hand is off the phone at the moment it takes the picture. Pressing the button was what blurred the shot.
A switch hands every spoken line to VoiceOver or TalkBack instead, so you hear it in your voice at your speed and nothing talks over anything.
Not a short list of favourites. Voices you install or enable yourself show up, and the list scrolls.
Full brightness on the first tap. No brightness menu to get through first.
Straight from your library, not just what the camera is pointed at.
Every control is labelled, and the labels say what the thing does, not what it's called.
Feature requests from the AppleVis community are credited by name in the commit history and in the source comments. If something is unlabelled, read in the wrong order, or just not useful — say so, and it gets fixed.
graph TB
A[📱 Camera Frame] --> B[🎯 YOLOv8 · 601 classes]
A --> C[📝 Vision OCR]
A --> D[🗣️ On-device Vision-Language Model]
A --> H[📏 ARKit LiDAR Depth]
B --> E[⚡ Metal]
C --> G[🌎 Offline Translation]
E --> F[🧠 Neural Engine]
D --> F
F --> I[🔊 Spoken Output · VoiceOver or in-app voice]
H --> I
D --> J[🔒 100% On-Device · nothing leaves the phone]
B --> J
| Component | Technology | Purpose |
|---|---|---|
| 🗣️ Narrator | LFM3-Omni-2B by Liquid AI (GGUF IQ2_XS + Q4_K_M vision projector, 735 MB, bundled) · llama.cpp (iOS 1.3.6 in review, Android 1.23 coming) | Scene and page description |
| 🤖 Detector | YOLOv8 (Ultralytics) | 601-class object detection |
| 🏗️ UI | SwiftUI (iOS) · Jetpack Compose (Android) | Native interface on both |
| ⚡ Acceleration | CoreML + Metal + Neural Engine · TensorFlow Lite | Hardware-optimized inference |
| 📊 Dataset | Open Images V7 | The 601 classes |
| 📏 Depth | ARKit LiDAR | Per-object distance |
| 🔋 Optimization | Thermal & battery aware, adaptive frame rate | Runs all day |
| Privacy Feature | Status | Description |
|---|---|---|
| 📊 Data Collection | ❌ NONE | Zero telemetry or analytics |
| 🌐 Internet Required | ❌ NO | Works in airplane mode |
| 📍 Location Tracking | ❌ NEVER | No GPS or location access |
| 🏢 Cloud Processing | ❌ NONE | 100% on-device AI, model included |
| 👤 Account Required | ❌ NONE | Install and use it |
🔐 Privacy is non-negotiable. Everything happens locally.
The one network call in the code: if a build does not have the narrator model inside it, the iPhone app offers a Download button and fetches it once, only when tapped (
VisionModelStore.swift). App Store and Google Play builds ship with the model, so they never make it.
ios/SwiftUI, CoreML, Metal and the Neural Engine. Full source, the Xcode project, and the YOLOv8 CoreML model.
Status: ✅ Live on the App Store, free.
android/The Android port: TensorFlow Lite YOLOv8 across the same 601 classes, with the model and the conversion tooling included.
Status: ✅ Live on Google Play, free.
android/# 1. Clone the repository
git clone https://github.com/nicedreamzapp/RealTimeAICam.git
# 2. Navigate to the iOS project
cd RealTimeAICam/ios
# 3. Open in Xcode
open "RealTime Ai Cam.xcodeproj"
# 4. Connect your iPhone and build!
In Xcode, pick your own signing team (and a bundle identifier of your own) under Signing & Capabilities before you run on a device.
The narrator weights are too large for git and are not in this repo. A build from source works for detection, reading and translation straight away; What's this? shows a Download button on the home screen and fetches the model (about 622 MB) the first time.
Android: the arm64 build links against prebuilt llama.cpp libraries and loads model files
that are also not in git, so build those first by following android/NARRATOR.md,
then open android/ in Android Studio.
NARRATOR.md)Replace the model file
ios/yolov8n_oiv7.mlpackage → YourCustomModel.mlpackage
Update the processor
YOLOv8Processor.swift if input/output shapes differ (it loads the generated yolov8n_oiv7 class and expects 601 classes)ios/class_names.txtYOLOv8Processor.swift too, or the filters will drop classes that no longer existLarge models
📝 Before submitting a PR, please include:
🎯 Areas where help is especially welcome:
The narrator here runs on llama.cpp on the phone. On the Mac I use MLX for claude-code-local, which runs Claude Code with local models and no API key.
The code is licensed under AGPL-3.0, the same license as the YOLOv8 detector it is built on. You can use, study, change and share it, as long as you share your changes under the same terms. The "RealTime AI Cam" name and app icon are not licensed, so a build you publish needs its own name and icon.
See LICENSE for the full text and NOTICE.md for the name, icon and third-party notices.
| Component | Creator | License |
|---|---|---|
| 🤖 YOLOv8 | © Ultralytics | AGPL-3.0 |
| 📊 Open Images V7 | CC BY 4.0 | |
| 🍎 CoreML, Metal, SwiftUI, ARKit | © Apple Inc. | Apple Developer Agreement |
| 🤖 TensorFlow Lite, Jetpack Compose | Apache 2.0 | |
| ♿ Feature requests & testing | The AppleVis community | with thanks |
| 🎨 App design & development | © NiceDreamzApps | AGPL-3.0 |
LiveOCRViewModel.swift), careful single-page reads (PageScanner.swift), mail and bill summaries in plain code with no model (MailSummarizer.swift), a second check on dollar amounts (MoneyCrossCheck.swift), offline Spanish to English (SpanishTranslationEngine.swift)HelpMeAim.swift), with unit tests in project 601Tests/ui/), the TensorFlow Lite detector (YoloDetector.kt), a JNI bridge to upstream llama.cpp that probes the GPU before trusting it (narrator_jni.cpp, NarratorEngine.kt), and the model conversion tooling (android/tools/)