ApexCompanion

A focus coach that watches your screen with Gemma 4, on your laptop, and says it out loud when you drift.

Demo video and landing page: https://kgorle1111.github.io/ApexCompanion/

Download

1. Download ApexCompanion.zip from Releases.
2. Unzip it and drag ApexCompanion into Applications.
3. Open it. macOS warns you because the app is not notarized: click Done, then go to System Settings → Privacy & Security and click Open Anyway (first time only; on macOS 15 and later, right-click → Open no longer gets you past this).
4. Pick your coach. It remembers your pick; to change it, choose Start with another persona… from the menu bar when no session is running.
5. Say what you're working on and pick a length: 25, 50 or 90 minutes, or no limit.
6. Look at your menu bar: ◉ 0m means it's watching. The number counts your focused minutes.

Step-by-step install and permissions guide: INSTALL.md.

Requirements: Apple Silicon Mac (M1 or later), macOS 14 or newer, 24 GB of RAM recommended (measured peak footprint 11.3 GB, so a 16 GB Mac works but swaps with a browser and editor open; 8 GB is not recommended), and about 12 GB of free disk (setup plus the 8.3 GB model; the launcher wants 10 GB still free after setup). First run needs internet: it downloads uv, Python with about 850 MB of packages, and the model. After that it makes no network calls.

It has no window: it lives in the menu bar and talks through notifications. Pause, mute, stop, or edit your sites from the menu bar (opening the app again still works too). It speaks out loud and names what's on your screen, so pick Mute voice in the menu bar (or run from source with --mute) in shared spaces.

Built for watching yourself: no accounts, no remote reporting. Install it only on your own Mac.

Developers can also run it from source.

The menu bar

The menu bar title shows what the coach is doing:

Title	Meaning
◉ 23m	in a session, 23 minutes focused
⏸	paused
in a call	a call app or meeting tab is in front, so it's not watching
🔇	voice muted
◉ Apex	idle, no session running

The menu:

• Working on: your goal and Focused 23 of 50 min
• Pause 15 minutes / Resume: while paused it takes no screenshots and runs no model.
• Mute voice / Unmute: silences the voice; notifications still show.
• Edit my sites…: opens your site list (see Your sites, your rules).
• Stop session: ends the session and shows your summary.
• Start a session… (when idle): asks for a goal and a length.
• Start with another persona… (when idle)
• Quit ApexCompanion

If your menu bar is crowded, macOS can hide the ◉ icon behind the camera notch on MacBooks. Quit a few menu bar apps, or use a bar manager, if you don't see it.

What it does

You start a session by saying what you're working on and how long (25, 50 or 90 minutes, or no limit). Every 10 seconds it checks what you're doing. If one of your site rules matches the frontmost app or the browser tab's URL, that decides. Otherwise Gemma 4 describes a screenshot and code decides whether you're working. Two distracted checks in a row and your coach speaks up: a one- or two-line roast, in character, that names what you're actually looking at and reminds you what you said you'd be working on. When the time is up, the session ends and shows your summary.

• Quiet while you work. No praise out loud: it used to speak up every ~5 minutes of focus, which interrupted the focus it's there to protect. Your focus shows in the end-of-session summary instead.
• Quiet during calls. While Zoom, Microsoft Teams, FaceTime, Webex, Around, Tuple, or a Google Meet, Zoom web or Teams meeting tab is in front, it takes no screenshot and says nothing.
• Your goal is for the coach, not the classifier. It shows in the menu and in roasts, but it never changes how the screen is classified, so the eval numbers below still apply.

Nothing leaves the machine while it runs. Once setup is done, pull the Wi-Fi and it keeps running.

Think of it as a gym coach who can see your screen but has signed an NDA with your laptop.

---

Why local

A screenshot of your screen is about the most private data you have: your messages, your bank tab, your half-written cover letter. Sending one to a cloud API every 10 seconds is not a reasonable thing to ask anyone to accept.

So the model runs on the laptop: Gemma 4 E4B, 8-bit, through mlx-vlm on Apple Silicon. HF_HUB_OFFLINE=1 keeps the Hugging Face client off the network, and gemma.py sets it by default too.

What stays on disk:

• Screenshots: nothing. Each frame goes to one temp file, gets classified, and is deleted right after its check.
• Session log: one local, text-only session-*.jsonl file per session, readable only by you (chmod 600). Each check records the category and verdict, the frontmost app name, the browser tab's host (not the full URL), which of your rules matched, and any line it spoke. The model's screen description is kept only for checks where a roast fired. Delete ~/Library/Application Support/ApexCompanion/logs (app) or ./logs (source) anytime.

Side effects of running locally: it's free, and after setup it works on a plane.

How it works

 every 10 s
 ┌───────────────┐   ┌──────────┐   ┌──────────────────────┐   ┌──────────────────┐
 │ screencapture │──▶│ sips     │──▶│ Gemma 4 E4B (vision) │──▶│ code maps        │
 │ -x -t jpg     │   │ -Z 1024  │   │ JSON: app, content,  │   │ category → state │
 └───────────────┘   └──────────┘   │ category             │   │ working /        │
                                    └──────────────────────┘   │ distracted       │
         screenshot deleted after the check                    └────────┬─────────┘
                                                                        │
             bad JSON or unknown category ──▶ "working" (never roast on garbage)
                                                                        │
                          distracted 2 checks in a row AND 60 s cooldown passed?
                                                                        │ yes
                                    ┌──────────────────────┐   ┌────────▼─────────┐
                                    │ macOS notification   │◀──│ Gemma 4 (text)   │
                                    │ + `say` (spoken)     │   │ ≤2-sentence roast│
                                    └──────────────────────┘   │ names app+content│
                                                               └──────────────────┘
 Ctrl-C ──▶ session stats: length, checks, % focused, call-outs

Before any of this, if a call app or meeting tab is in front, the check is skipped: no screenshot, no model call, no line spoken. Pause skips checks the same way.

Before the vision call, your site rules check the frontmost app and the active tab's URL. On a rule hit, the vision model is skipped for that check (see Your sites, your rules).

One model load, reused for both the vision call and the roast. The same Gemma 4 weights do both jobs.

Category map (lives in code, not in the prompt):

Category	State
coding, docs_reading, notes_writing, educational_video, messaging, news_reading, other	working
entertainment_video, social_media, shopping, gaming, feed_browsing, sports	distracted

Messaging, news, and "other" count as working on purpose. A false roast while you're replying to your professor is worse than a missed one. The prompt also tells the model to ignore ads, banners, and pop-ups, so a sidebar ad doesn't decide the category.

Trade-off to know about: "other" fails open to working. A native game that lands in "other" won't be roasted, unless you put it on your list (next section).

Your sites, your rules

The model's guess is a default, not the last word. You keep a plain-text list, sites.txt, created on first run, with two sections:

[distracting]
reddit
steam
[ok]
github
figma

• One site or app per line. Not case-sensitive, whole-word match, # starts a comment.
• "ok" always wins. If something matches both sections, it counts as working.
• Edits apply on the next check. No restart.
• What it matches against: only the frontmost app's name and, for Safari, Chrome, Arc, Brave, and Edge, the URL of the active tab (macOS asks once to let ApexCompanion control your browser; it reads only the tab's address, locally). In Firefox and other browsers, only the app name. It never matches on-screen text, so a page can't trip or dodge a rule by what it displays.
• That means rules work by domain or app, not by YouTube channel: allow a site or an app, not one channel on it.
• Only the address counts, not the search box: googling "reddit" is not a visit to reddit.
• Firefox exposes no tab address, so site rules don't work there; app-name rules (like steam or firefox itself) still do.
• Built in: youtube.com/shorts always counts as distracting. Your [ok] list still wins.
• On a rule hit, the vision model is skipped for that check.
• A rule changes the verdict, not the timing: it still takes 2 distracted checks in a row before a roast.

The design point: your list is deterministic and overrides the model both ways, so a wrong guess about your own sites is something you fix in one line, not something you argue with.

Personas

Two original archetypes, not impressions of any real person or character, picked with --persona:

• The Closer (--persona closer): an arrogant, razor-witty dealmaker who treats you like the associate about to blow the deal.
• The Whip (--persona whip): a cold, slow-talking political operator who speaks in unsettling metaphors.

Each persona is a prompt, a few example lines, and a built-in macOS voice, in personas.py. Repeat offenses escalate: the second call-out in a session is sharper than the first, and from the third on it counts out loud. Praise is off by default (--praise-every 0), so a focused stretch stays quiet; --praise-every N brings it back after N working checks in a row. Roasts can also mention the goal you set ("you said you'd be working on…"). Every spoken line passes a word filter: if a line contains a slur or injected instructions, it's replaced with a fixed safe line.

Tech stack

Layer	What	Why this
Model	Gemma 4 E4B, 8-bit MLX build (mlx-community/gemma-4-e4b-it-8bit)	Native image input, fits in about 11 GB, sub-second vision check; the same weights write the roast
Inference	MLX 0.32.4 + mlx-vlm 0.7.10 on the Apple GPU (Metal)	Fastest local path on Apple Silicon; no server, no network
Coach	Python 3.12, standard library first: subprocess, json, urllib.parse, tempfile, resource	Small, auditable loop; every outside call is a subprocess with a timeout
Screen and voice	macOS built-ins: screencapture, sips, osascript (notifications, browser tab address), lsappinfo (front app), say	Nothing to install; works offline
App shell	Swift + AppKit host (packaging/main.swift), menu-bar status item, ad-hoc signed .app	A compiled host is what macOS grants Screen Recording to, and it stays alive as the coach's parent
Installer	Bash launcher (packaging/launcher.sh) + uv, pinned dependencies (packaging/constraints.txt)	One-click first run: private Python, packages and model in Application Support
Quality	pytest (stubbed model, about 1.7 s for the suite), ruff 0.14, eval.py on real web pages with false-roast and recall gates	Fast tests on every change; the real-model eval is the release gate
CI	GitHub Actions: ruff on Ubuntu, tests on macOS 14 (Apple Silicon)	MLX only ships Apple Silicon wheels
Site	One static HTML file, no JavaScript framework, no third-party requests; GitHub Pages from gh-pages	Loads with Wi-Fi off; nothing to break

Architecture

Processes

flowchart LR
    user([You]) -->|double-click| host
    subgraph app["ApexCompanion.app"]
        host["Swift host<br/>main.swift<br/>menu-bar item"]
        launcher["launcher.sh<br/>first-run setup,<br/>persona, goal"]
    end
    host -->|spawns, stays parent| launcher
    launcher -->|exec| coach["Python coach<br/>apex.py"]
    coach -->|writes every check| state[("state.json")]
    host -->|polls every 2 s| state
    host -->|Pause / Mute / Stop| control[("control.json")]
    coach -->|reloads on change| control
    coach -->|appends| logs[("logs/session-*.jsonl")]
    coach --> model["Gemma 4 E4B<br/>MLX on the GPU"]

macOS grants Screen Recording to the app that launched a process, and only while it runs. That is why the Swift host stays alive as the parent of the coach instead of handing off to a detached script. Everything the processes share is a file in ~/Library/Application Support/ApexCompanion/, readable only by you.

One check

flowchart TD
    tick([every 10 s]) --> ctl{"control.json:<br/>paused or stop?"}
    ctl -->|stop| endS([end session, write summary])
    ctl -->|paused| idle["no screenshot, no model"]
    ctl -->|no| call{"call app or<br/>meeting tab in front?"}
    call -->|yes| quiet["quiet: no screenshot"]
    call -->|no| rule{"your site rules<br/>match app or tab address?"}
    rule -->|yes| verdict
    rule -->|no| shot["screencapture → sips 1024 px<br/>(temp file, deleted in finally)"]
    shot --> vlm["Gemma 4 describes:<br/>app, content, category"]
    vlm --> map["code maps category → working / distracted<br/>unparsed → working"]
    map --> verdict["verdict"]
    verdict --> coachS{"2 distracted in a row<br/>and 60 s cooldown?"}
    coachS -->|no| log
    coachS -->|yes| roast["Gemma 4 writes a roast<br/>(goal + what was on screen)"]
    roast --> guard["word filter: slur or injected<br/>instruction → fixed safe line"]
    guard --> speak["notification + say<br/>(say skipped when muted)"]
    speak --> log["log event + state.json"]

Trust boundaries

Input	Trusted?	How it's handled
Pixels on screen	No	The model only describes them; it never decides the verdict. Its text never reaches a shell or AppleScript as code: notifications pass it as an argument, say gets it after --.
Model's description	No	Mapped to a verdict by a fixed table in code. Never used to match your site rules. Reaches the roast prompt capped at 120 characters.
Model's roast	No	Cleaned to two sentences and checked by a word filter before it is shown or spoken.
Front app name, tab address	Yes (from macOS)	The only inputs your site rules match. Logs keep the host, not the full address.
Your goal and sites list	Yours	Goal is used in the roast only, never the classifier, so accuracy numbers don't move with it.

Files on your Mac

Path	Written by	What
app/, venv/, python/	launcher	The coach code and its private Python
sites.txt, config	you, launcher	Your lists, persona, last session length
state.json	coach	Live state for the menu bar
control.json	menu bar	Pause, mute, stop requests
logs/session-*.jsonl	coach	One file per session, deleted after 7 days
Model weights	launcher (once)	Hugging Face cache, about 8.3 GB

Repo map

File	Role
apex.py	The loop: capture, rules, classify, coach, speak, log
gemma.py	Loads Gemma 4 once; classify() and generate_text()
rules.py	Your site lists, front app, tab address, Shorts rule, meeting detection
personas.py	The two coaches, roast prompts, output cleaning and word filter
control.py	state.json / control.json reading and atomic writes
report.py	Session summary from the logs
eval.py + evals/	Accuracy harness; labels and results (screenshots are not committed)
packaging/	Swift host, launcher, build script, pinned dependencies
site/	The landing page
tests/	Unit and loop tests with a stubbed model

Design decision: the model never says "distracted"

The first version asked Gemma one question: is this screenshot working or distracted? It labeled a YouTube cat compilation (a synthetic test image) "working". Gemma 4 did it, and so did Gemma 3 when we tried it as a cross-check.

A binary label gives the model room to rationalize ("maybe they're researching cats"). So we split the job:

1. The model describes. It returns {"app", "content", "category"}: which app, what's on screen, and one category from a fixed list.
2. Code decides. A lookup table maps the category to working or distracted. The model never emits the verdict.

With describe-then-categorize, the same cat video landed in a distracted category, so the code called it distracted.

That fix started as one synthetic test case. The real-page eval came after: on a blind batch of 23 real public web pages captured after tuning, the classifier got 15/16 clear-cut pages right (Wilson 95% interval about 72% to 99%), with 0 false roasts on clear-cut pages (details under Evaluating it).

Measured A/B. We later ran a yes/no prompt against the shipped pipeline on the same 41 clear-cut real pages (same model, temperature 0; evals/binary_baseline.py, results in evals/results_binary_baseline.json). The headline yes/no prompt lists the same working and distracted kinds of screen as the shipped category map, so the comparison is mostly about who says the verdict (the shipped prompt also adds a few one-line definitions, such as what counts as a feed):

Clear-cut pages	Correct	Distractions caught	False roasts	Correct, blind batch only
Shipped (describe, then code decides)	40/41	17/18	0	15/16
Yes/no, same rubric as shipped	29/41	6/18	0	9/16 (0/7 distractions caught)
Yes/no, original narrow rubric¹	27/41	5/18	1	11/16

¹ Names only entertainment and social media; its false roast was a CNN page with a pop-up. Run with --prompt original.

Even with the full rubric, the yes/no prompt often described the screen correctly and still said working: the Steam store came back "working" with the reason "Reading game store page for Hades". Other misses echoed the rubric instead of describing the page ("Coding, reading docs, and notes visible" on Reddit). Caveats: the shipped prompt was tuned on batch 1, so the blind-batch column is the fair comparison, and it is only 16 pages.

It's the pattern we default to: deterministic guards around one structured model call, and code owns the final decision. The debounce, cooldown, and fail-open default are plain Python (Coach in apex.py, no I/O) with unit tests in tests/. Your site list (rules.py) works the same way: plain code, checked before the model call, which it skips on a hit.

Prior art (and what's different)

We're not the first to point a local model at a screen to catch drift. AccountyCat is an MIT macOS focus companion running local Qwen via llama.cpp; it reads the app and window title first and uses a screenshot only when that's ambiguous, then nudges you. Observer AI lets you build local screen-watching agents, including "when I'm distracted, call me". Dayflow keeps a local screenshot timeline without intervening.

What ApexCompanion does differently:

• Gemma 4 vision on every check your own rules don't already decide, not window-title text with a screenshot fallback.
• Spoken persona roasts that name what's on screen, instead of a quiet nudge.
• Code owns the verdict. The model describes; a lookup table decides.

The trade-off is real: a text-first design like AccountyCat's is cheaper on compute and battery than running a vision model every 10 seconds.

Run from source

Requires an Apple Silicon Mac (MLX), Python 3.12, and uv.

uv venv --python 3.12
uv pip install mlx-vlm pillow

Download the model once while you're online (it goes into the Hugging Face cache):

.venv/bin/python -c "from huggingface_hub import snapshot_download; snapshot_download('mlx-community/gemma-4-e4b-it-8bit')"

Grant screen access: System Settings → Privacy & Security → Screen Recording, enable your terminal app, then quit and reopen it. Without this, screenshots come back as just your wallpaper and everything looks like "other", so the coach would silently stay quiet. apex.py checks for this and refuses to start without the permission. It also warms the model at boot, so the first real check isn't slow.

Run:

HF_HUB_OFFLINE=1 .venv/bin/python apex.py --persona closer --goal "finish the lab report" --minutes 50

Swap closer for whip to change the voice. Turn your volume up. Ctrl-C ends the session and prints your stats.

Useful flags: --interval (seconds between checks, default 10), --streak (distracted checks before a roast, default 2), --cooldown (seconds between roasts, default 60), --goal (what you're working on; shown in the menu and used in roasts, never in classification), --minutes (session length; the session ends and prints your summary when it's up), --state-dir (where state.json and control.json live, for the menu bar), --praise-every (spoken praise after N working checks in a row; default 0, off), --mute, --once (one check, print, exit), --image FILE (classify a file instead of the screen; repeatable), --sites FILE (path to your site list; defaults to sites.txt next to apex.py).

Measured numbers

Measured on an M3 Max, mlx-community/gemma-4-e4b-it-8bit, mlx-vlm 0.7.10, offline.

What	Number	Caveat
Model load	3.4 to 3.7 s
First classify call	2.6 s	includes warm-up
Vision classify call only, warm	p50 0.87 s, p95 0.96 s (batch 1); p50 0.78 s, p95 0.94 s (batch 2)	model already loaded; excludes screencapture and resize
Roast generation	0.57 to 0.79 s
Peak memory	11.3 GB footprint	/usr/bin/time -l over a 306-check soak; steady state 8.6–8.9 GB, no growth seen in 5 min
End-to-end check	2 distracted checks → spoken roast	ran end to end on a real desktop in the 52-minute live soak below (3 roasts)
Live soak (one real session on the author's own screen, Oct 24)	52 min, 624 checks at a 5 s interval (default is 10 s), 0 errors, 0 unparseable outputs	reliability soak, not an accuracy test: the checks are unlabeled; source: evals/live_session_2026-10-24.json
Live soak, end to end per check	p50 1.29 s, p95 1.60 s	rules + screencapture + classify; n = 624
Live soak, vision classify	p50 0.97 s, p95 1.26 s	n = 624
Live soak, capture	0.18 s p50	screencapture + resize
Live soak, roasts	3	the author opened cat videos to test it
Batch 2, blind (23 pages, never tuned on)	15/16 clear-cut (Wilson 95% interval about 72% to 99%), 21/23 overall, 0 false roasts on clear-cut pages	the headline; captured after prompt tuning finished; labels hand-checked by the author (Oct 24), not an independent reviewer
Batch 2 distracted recall, clear-cut	6/7

Fixed settings (configuration, not measurements): 10 s check interval, 2 consecutive distracted checks before a roast, 60 s cooldown between roasts, screenshots downscaled to 1024 px on the long side.

Ceilings

Where it breaks, for the one person it runs for. Full gap table and what's still unverified: HARDENING.md.

Metric	Value	How we know
Coaches per Mac	1 (a second one refuses to start)	single-instance lock; two would need about 22 GB
Break point	memory: 11.3 GB peak footprint, so 16 GB Macs are tight and 24 GB is recommended	live, /usr/bin/time -l over a 306-check soak; 8 GB and 16 GB Macs not tested
Time per check, live	p50 1.29 s, p95 1.60 s	live, 52-minute soak, n = 624
Shortest useful interval	about 1.6 s; below that, checks run back to back (the loop never overlaps itself)	derived from the live p95
Cost per check	$0, no network calls after setup	on-device by design
Largest input	one screenshot, downscaled to 1024 px on the long side; model output capped at 100 tokens (classify) and 80 (roast)	configuration
When it gives up	6 failed checks in a row: it stops, says why, and shows ⚠︎ instead of pretending you're focused	broken-coach circuit breaker, unit-tested
Log growth	session logs deleted after 7 days; apex.log rotated at 1 MB	configuration

Evaluating it

eval.py runs the classifier over real screenshots in evals/ against hand labels (evals/labels.json, evals/labels2.json) and prints accuracy and per-case latency. The repo has the labels and results (evals/results*.json), but not the screenshots: they're captures of third-party pages, so they're git-ignored. That means you can't rerun the eval from the repo alone.

Pages a reasonable person could call either way are labeled ambiguous and reported separately. All runs: Gemma 4 E4B 8-bit, mlx-vlm 0.7.10, M3 Max, offline.

Batch 2 (the headline): 23 pages, blind. Captured after prompt tuning had finished, and never used to tune anything. The labels, including which pages are ambiguous, were hand-checked by the author (Oct 24); no independent reviewer has checked them.

• Clear-cut pages: 15/16 (Wilson 95% interval about 72% to 99%), with 0 false roasts on clear-cut pages.
• All 23 pages: 21/23.
• Distracted recall: 6/7, exactly at the 0.85 gate.
• Misses: a YouTube Shorts basketball clip filed as educational_video (a clear-cut distraction it let through), and a celebrity-gossip page (Page Six) filed as news_reading (labeled ambiguous). Both failed open to working.
• Vision classify latency: p50 0.78 s, p95 0.94 s, warm.

Batch 1 (the tuning set): 28 pages, used while tuning the prompt. Real public web pages captured in a headless browser: 12 working, 11 distracted, 5 hard cases. The author confirmed every label. Because we tuned on it, it's not a fair test on its own.

• Clear-cut pages: 25/25.
• All 28 pages: 26/28. Both misses were ambiguous cases: a lofi "study" stream whose player showed "Video unavailable", and a random Wikipedia biography. The lofi stream was called distracted: that is the one false roast in the eval, on a page labeled ambiguous.
• Distracted recall: 11/11.
• Vision classify latency: p50 0.87 s, p95 0.96 s, warm.

Pooled: 40/41 clear-cut across 51 real pages (95% interval lower bound about 87%). That is a small set, not a general accuracy figure.

Checking that the eval can fail. We broke the category map on purpose and re-ran it (mutation testing). Moving entertainment_video into the working list turned the eval red on the distracted-recall gate (threshold 0.85). Moving coding into the distracted list turned it red with 6 false roasts. The first version of the gate checked accuracy only, and it did not catch the first mutation: with entertainment_video mapped to working, accuracy was still 21/23 (91.3%), so the eval passed. We added the recall gate; on the same mutation, distracted recall was 9/11 (82%), under the 0.85 bar, so it went red. Details in evals/iterations.md.

Logs and observability

Everything stays on your Mac. Each session writes one owner-only JSON Lines file (logs/ when run from source, ~/Library/Application Support/ApexCompanion/logs/ for the app). Files older than 7 days are deleted at startup (--log-days).

Every line carries a session id, a schema version and an event:

Event	What it records
session_start	persona, interval, cooldown, replay or screen mode, Python and macOS versions
model_ready	model id, load time, memory
check	verdict, category, rule hit, site host (never the full address), per-stage timing (rules, capture, classify, roast, total), whether the word filter replaced the line; screen text only when a roast fired
error	which stage failed (rules, capture, classify, roast) and the error; the session keeps going. After 6 failed checks in a row it logs stage breaker, stops, and exits with code 3
warning	the warm-up answer didn't parse (the model loaded but may be answering badly)
session_end	checks, roasts, errors, focus %, duration, peak memory

To read it:

.venv/bin/python report.py            # latest session
.venv/bin/python report.py --all      # every session
.venv/bin/python report.py --json     # for scripts

The report shows focus %, roasts, how often a site rule decided the check, how often the word filter fired, error and unparsed-output rates, and p50/p95 for each stage. The app shows the same summary when you stop it or the session time runs out. This is also where real end-to-end latency comes from: the 0.87 s figure above is the classify step alone, and total in a live session includes the screenshot.

The menu bar and the coach talk through two small local files, state.json and control.json, in ~/Library/Application Support/ApexCompanion (owner-only; --state-dir changes the folder). Nothing leaves the Mac.

Limitations

• Web pages only in the accuracy eval. The accuracy eval is 51 real web pages captured in a headless browser: no native-app frames (VS Code, the terminal, the native Spotify app, games) and no desktop frames. The 52-minute live soak did run on a real desktop, but its checks are unlabeled, so it measures reliability and latency, not accuracy. Your site list can cover a native app by name.
• Fail-open categories hide misses. Anything filed under "other", or misfiled into a working category, counts as working and is never roasted. In the blind batch, a Shorts clip went to educational_video and a Page Six gossip page to news_reading. The built-in Shorts rule now covers the first case in supported browsers.
• Small eval, one labeler. 16 blind clear-cut cases is a small set; one miss is about 6 points. The labeler is the author: batch-2 labels were hand-checked by the author (Oct 24), but no independent reviewer has checked either batch.
• Not reproducible from the repo. Labels and results are committed; the screenshots are not (third-party pages).
• Heavy for a background app. 11.3 GB peak memory footprint, and a vision call every 10 seconds. Expect trouble on Macs with little unified memory, and expect it to cost more battery than a text-first approach.
• Mac only. screencapture, sips, osascript, say, and MLX are all macOS / Apple Silicon.
• It sees one screen, once every 10 seconds. It can miss short distractions, and on multiple displays it only judges what screencapture captures.
• Context is a single frame. A lecture on YouTube with a cat thumbnail in the sidebar, or gameplay footage you're watching for work, can be misread. Fail-open defaults and the 2-check debounce reduce false roasts; they don't eliminate them.
• The session log is a record of what you looked at. Text only, local, readable only by you, but it includes app names and site hosts. Delete the logs folder if you don't want it.
• Spoken roasts name what's on screen. Anyone nearby hears it. Use --mute (or mute your Mac) in shared spaces.
• Rules are word matches. A rule like news matches any app name or tab URL containing that word. Keep entries specific.
• It's a nudge, not a blocker. It won't stop you from opening anything. That's deliberate, but it means it only works if you leave it running.
• No automatic fallback. The Ollama build of Gemma 4 we had takes no image input, which is why vision runs through . Gemma 3 vision via Ollama worked in a quick test, but that path isn't wired into .

What's next

• A one-click "That was work" correction from the notification.
• Skip the vision model when the front app or page hasn't changed: fewer model calls and better battery, with battery drain measured.
• Capture every display, not just the main one.
• Evaluate on real desktop and native-app screenshots, labeled by someone other than the author.
• A domain-only baseline, to show what vision adds over site rules alone.
• A notarized build with bundled Python, for a shorter first run.

License

MIT for the code in this repo. The Gemma 4 weights are distributed by Google under their own terms; check the model card before redistributing them.