Notes:
- The 2.3.0-GA release branch is at https://github.com/ancientcomputing/locallm/tree/2.3.0-GA
- The project website is at https://thisbrain.ai/locallm/
LocalLM Lab
LocalLM Lab is about running AI on your own Mac instead of renting it
from a hosted provider — no per-token bill, no data leaving your machine, no
dependency on someone else's uptime. It ships two ways to get there: an app,
and an SDK.
The app (macOS 26+, Apple Silicon, Apple Intelligence enabled) turns your
Mac into a private chatbot with an OpenAI-compatible API — fully offline with
Apple's on-device model or open-weight models you download and run locally,
or pointed at a hosted provider when you want one — and gives other apps and
scripts a way to call whichever model you've picked.
It's a single Dock app with five panels:
- Prompt Playground — a chat window over the model you've selected.
- AI Models — add and switch between Apple's on-device model, Private Cloud
Compute, open-weight models you download and run locally via MLX, and hosted
providers (Claude, GPT, OpenRouter, or any OpenAI-compatible server, with
provider-native web search).
- Connectors — System Clock, Filesystem, Weather, Calendar, Reminders,
Contacts, and Location access the model can use as tools, each gated by your
permission.
- MCP Servers — connect Model Context Protocol servers (tool discovery,
OAuth, Keychain-backed tokens) and hand their tools to the model.
- API Lab — an OpenAI-compatible
/v1 endpoint served by the app itself
(localhost, plus optional LAN HTTPS with a self-signed cert). Point any
OpenAI-SDK tool at your Mac.
It began local-only — hence the name — and 1.0 added the hosted providers behind
the same interface, so "local" is now the default rather than the only option.
On macOS 26 only Apple's on-device model runs; open-weight (MLX) models and the
hosted providers need macOS 27. Private Cloud Compute will be in the Mac App
Store version: it needs Apple's Private Cloud Compute entitlement, which the DMG
doesn't carry.
Product page: thisbrain.ai/locallm — or
get the app directly:
LocalLM Lab 2.3.0-GA (arm64 DMG).
LocalLM Lab SDK
The SDK is the same engine LocalLM Lab runs on, published as binary
xcframeworks you link into your own native macOS app — so your app can run
local AI (Apple's on-device model, or an open-weight model you ship or let the
user download) instead of routing every request through a paid host API,
with hosted providers still available behind the same interface for when you
want one. LocalLMLabSDKCore carries the
connectors, a full MCP client, Core's Workspace (filesystem) tools, and the 1.0
model layer — routing and residency across every provider behind one API, with
SystemModelProvider (Apple on-device) built in. Add the provider modules you
need: LocalLMLabSDKInference (open-weight models via MLX),
LocalLMLabSDKClaude (Claude through the Foundation Models interface, via ClaudeForFoundationModels),
LocalLMLabSDKRemote (hosted providers — GPT, Claude's online API, OpenRouter,
any OpenAI-compatible server), and LocalLMLabSDKComponents (prebuilt SwiftUI).
Proven under App Sandbox, with a signed path to both Developer ID distribution
and the Mac App Store — it is not a demo dependency, LocalLM Lab itself runs on
it. Full developer guide:
docs/sdk-guide.md. Deeper walkthrough of the SDK and its
reference apps is further down.
About this repo
ancientcomputing/locallm is the public home for everything shipped to LocalLM
Lab users and SDK developers. The app and SDK source stay closed-source;
what's public here is:
- SDK releases — the SwiftPM consumption manifests, binary
xcframeworks (one GitHub Release per version, tag
v<version>), the developer
guide, and a machine-checked API surface.
- examples/ — a dozen small, runnable SDK reference apps (one
per integration path), plus scripts for the
localai-cli toolkit and the
OpenAI-compatible API Lab endpoint. Full list with descriptions in the
Building on the SDK section below.
- toolkit/ — the
localai-cli toolkit (zip + .sha256).
0.6–1.0.0-beta.3 are checked in here; 1.0.0-beta.4 through 1.0.0-RC.1
shipped as a release asset on
ancientcomputing/locallm-releases;
from 1.0.0-GA on it ships as a release asset on this repo instead,
alongside the app DMG and the SDK xcframeworks. See that folder's README to
download and verify; full CLI reference at
thisbrain.ai/locallm/cli.html.
- Components/ —
LocalLMLabSDKComponents, prebuilt SwiftUI for
managing MCP servers and models, built on the SDK's public API.
- MCPAppsHost/ —
LocalLMLabSDKMCPAppsHost (2.0), host-side support for
MCP Apps: the sandboxed view for a server's ui:// widgets, the widget↔host bridge, and the
widget lifecycle. Source, built on the SDK's public API like .
Building on the SDK
The model layer builds for macOS 26 & 27. SystemModelProvider works on
macOS 26; MLX, ClaudeForFoundationModels, the hosted providers, and Private Cloud Compute
need macOS 27. Coming from 1.x? See docs/migrating-to-2.0.md:
2.0 is a one-time breaking release (mostly tool authorization), then source compatible for every
2.x. From 0.8.x, start with docs/migrating-to-1.0.md.
Two ways to turn a connector or MCP server into something the on-device model can actually call as
a tool. Path A: drop in a ready-made Tool Core already ships for it —
GetUpcomingEventsTool, SearchContactsTool, MCPTool (built at runtime from any MCP server's
own schema), and the rest — correctness lessons from real, observed on-device model failures
already baked into their descriptions. Path B: hand-write your own adapter directly against
the underlying connector call (CalendarAccess, MCPServerManager, etc.) for full control over
tool names, schemas, and descriptions. Neither is the "real" one — both ship in Core, and an app
can mix them. See docs/sdk-guide.md §7a
for the full framing.
- Decision models (Jev), new in 2.0 —
lab.decide asks typed questions (yes / no, one of
several, a scale) and gets one allowed answer each with probabilities, no text, from TypeSafe's
Jev, Featherless or OpenJev on a local MLX model; a small decider can stay loaded next to
your chat model. Start with Decision models (Jev) in your app,
then docs/sdk-guide.md §6c and
examples/jevdk/.
- docs/sdk-guide.md — the full developer guide: linking Core, the model
layer (§6a), entitlements, all three MCP auth types, Keychain storage, App Sandbox/MAS signing,
ready-made vs. hand-written tool-calling (§7a), and a full function/type reference (§12).
- docs/mcp-diagnostics.md — the MCP client's logging: one
verbosity knob, Apple's unified log, and an exportable buffer for user bug reports.
- docs/migrating-to-1.0.md —
0.8.x → 1.0 (macOS 27).
- docs/api-surface.md — machine-generated public API list (the check
behind §12).
- docs/tested-models.md — a point-in-time snapshot of which
open-weight (MLX) models actually tool-call, and why several don't.
- CHANGELOG.md — the public SDK surface, version by version.
- — a repo-local Agent Skill
for developers using Codex or Claude Code to build macOS Swift apps with this SDK. Codex
discovers it through ; Claude Code discovers it through
.
Reference apps
Roughly simplest to fullest — every one runnable, with full annotated source in
docs/annotated-examples.md:
- examples/repo-qa/ — a minimal CLI: Apple's on-device model calling a
real no-auth MCP server's (Deepwiki's) tools, built straight from their live schema (Path A).
- examples/os-matrix/ — one CLI binary that runs on macOS 26 and
27 with no source
#if, model families gated by OS at registration
(ModelAvailability.requiresOS).
- examples/repo-qa-local/ —
repo-qa again, but the answer comes
from a downloaded open-weight MLX model. The smallest model-layer + LocalLMLabSDKInference
example.
- examples/components-demo/ — a working "add / manage MCP
servers" screen assembled from prebuilt
Components views, no MCP UI hand-written.
- examples/plate-today-tools/ — Calendar + Reminders + the
Todoist (OAuth MCP) server → a spoken-language day summary, on Core's ready-made tools (Path A).
- examples/plate-today/ — the same day summary, built with
hand-written
Tool adapters (Path B). Diff the two to see exactly what changes.
- examples/workspace-buddy/ — sandboxed AI edits to a
user-picked folder via Core's
WorkspaceTools, on-device model, a security-scoped bookmark
that survives relaunch.
- examples/workspace-buddy-local/ —
workspace-buddy +
a downloaded MLX model, running the model layer the App Sandbox, streaming.
Plus examples/api-lab/ (scripts + a chat app for the OpenAI-compatible
endpoint) and examples/localai-cli/ / localai-cli-swift/
(calling the localai-cli toolkit directly), which don't link the SDK.
Roadmap
If you want to see a new feature in LocalLM Lab, please feel free to raise an issue.
