Larkmere AI: automating the system that has no API

Many business systems have no API, and AI output can be confidently wrong. This project is a working answer to both: a robot that operates a desktop app purely through the screen and stops rather than guesses, and code that checks AI output instead of trusting it.

It's built around Larkmere AI, an invented company selling model API access to businesses. Its customers, staff, model tiers, prices and history are all made up, and it's seeded data all the way through.

Live demo: https://sarthaksansar.github.io/larkmere-ai/

Why I built it

Getting AI into a real business is rarely blocked by the model. It's blocked by old systems with no way in, and by the fact that nobody can sign off on output they can't verify. I wanted a project that shows that work end to end, in code anyone can run, with nothing confidential in it.

What this project shows

• It can drive software that offers no way in. It reads the screen, confirms where it is, and acts like a person would.
• It knows when to stop. A look-alike production window appears mid-run, and the robot stops with no further clicks or keystrokes.
• It doesn't take a reading's word for it. Plain code recomputes every line and total on every statement. Bad figures go to a person.
• People stay in charge of what gets written. No note reaches the CRM until a named person approves it.
• It speaks the business's numbers. New business, expansion, churn, retention, committed spend against overage, and movement between model tiers.

Try it

Needs Python 3.11 or newer. A virtual environment is a good idea.

pip install .

python -m larkmere demo

This generates the sample company, opens a page where you review the drafted notes, then asks before the robot takes your mouse and keyboard for a few minutes. To stop it at any time, push the mouse into a screen corner. It finishes by opening the results page.

Option	What it does
--no-robot	Skips everything that uses the mouse and keyboard
--approve-as "Name"	Skips the review page: approves clean notes and rejects flagged ones under that name
--mode offline	Uses rule-based stand-ins in place of the recorded Claude responses, so no AI output at all
--mode live	Calls Claude for real. Needs pip install .[live] and a Claude API key in ANTHROPIC_API_KEY

To click around the CRM yourself, run python -m larkmere crm. On Linux you also need a display and these packages: python3-tk libgl1 libglib2.0-0 fonts-dejavu-core, plus xvfb for a virtual display.

To run the tests, use pip install .[dev], then python -m pytest. The tests that take over the mouse are skipped on Windows and Mac unless you set LARKMERE_GUI_TESTS=1.

Platforms

Platform	What the project does with it	How far it's been tested
Claude API	Reads statement screenshots and drafts notes	One full live run, on Windows, using claude-opus-5-5. Those responses are committed and the demo replays them
Microsoft Teams	Imports a meeting transcript from a downloaded .vtt file	Sample files written for this project, not a real export
Granola	Imports transcript text, mapping "Me" and "Them" to real names	Sample files written for this project, not a real export

The Teams and Granola importers read files. They don't connect to either service, so nothing is fetched or posted automatically. This is an independent demo, not affiliated with or endorsed by any of these companies.

What is and isn't AI

Being straight about this matters, so here it is step by step.

Step	How it works
Reading the screen	A local OCR engine: a small neural network running on your machine, with no cloud involved
Deciding where the robot is, and whether to act	Rules only. No model is ever asked whether it's safe to click
Turning a statement into data	Claude, through the Claude API. The default demo replays a recorded run. --mode offline swaps in a rule-based parser
Checking the figures	Arithmetic only
Drafting a note	Claude, through the Claude API. The default demo replays a recorded run. --mode offline swaps in a cue-phrase picker
Checking a note's facts	Rules only: figures, dates, names and citations must match the transcript
Approving a note	A person
Revenue metrics	Arithmetic on seeded data

The default demo needs no key and makes no API calls. The robot and every check run for real, but Claude's answers are played back from one recorded live run. So a replayed statement's figures come from that run's screenshots, not the ones taken as you watch. In live mode, statement screenshots and transcripts are sent to the API. They're all fake.

In the recorded run, Claude transcribed all six statements and passed the planted errors through exactly as displayed, without correcting them, which is what the checks need. Each clean statement verified, and each planted fault was quarantined. It also recorded a transcript's "ignore previous instructions" line as a risk and didn't obey it. That's one run on six statements and four meetings, so it shows the design works and isn't an accuracy figure.

This is the beginning of the README. Read the rest at the original. Read the full README ↗