MLXUI is a Local AI Browser for MLX. Run MLX models AI Workflows.
저장소 README에서 가져옴
README
README 전체를 불러오는 중입니다. 아래는 저장된 발췌입니다.
Run MLX models visually — no terminal, no Python, no command-line flags.
MLXUI is the SwiftUI user interface layer for MLX models — a native macOS app built on
mlx-swift that lets you browse, install, and
run local AI models on Apple Silicon. It ships a curated catalog of MLX-optimized models
from Hugging Face's mlx-community, downloads
them with one click, and provides a purpose-built Run UI for each model type.
The mission: become the UI layer for every MLX model on Hugging Face — the place
where anyone on a Mac can discover, download, and try out local AI without ever opening
a terminal.
Run MLX models visually — no terminal, no Python, no command-line flags.
MLXUI is the SwiftUI user interface layer for MLX models — a native macOS app built on
mlx-swift that lets you browse, install, and
run local AI models on Apple Silicon. It ships a curated catalog of MLX-optimized models
from Hugging Face's mlx-community, downloads
them with one click, and provides a purpose-built Run UI for each model type.
The mission: become the UI layer for every MLX model on Hugging Face — the place
where anyone on a Mac can discover, download, and try out local AI without ever opening
a terminal.
What this is
The ultimate native SwiftUI cockpit for local AI, built exclusively for MLX and Apple Silicon.
There are thousands of MLX models on Hugging Face. Using them today means finding the
right repo, reading the README, managing Python dependencies, figuring out CLI flags,
and converting formats when things don't match.
MLXUI replaces all of that with a single app. Browse by category (Chat & Text, Vision,
OCR, Speech-to-Text, Text-to-Speech, Embeddings), filter by what fits your hardware,
install with a click, and run each model in a purpose-built interface — chat for LLMs,
a microphone and transcript for ASR, a voice picker for TTS, an image dropwell for OCR.
Under the hood, MLXUI wraps Apple's mlx-swift and mlx-libraries (MLXLLM, MLXVLM,
MLXWhisper, MLXAudioTTS, etc.) in a modular SwiftUI architecture. Each model type gets
its own isolated module folder — updating Whisper never touches Kokoro, and a
contributor adding image-generation support doesn't need to understand the chat pipeline.
Screenshots
Browse
Model Detail
🔀 Run AI Workflows
What is mlx-workflow?
A readable AI workflow you can run locally — and MLXUI now runs it natively.
1. Read Audio client-call.m4a
2. Transcribe Whisper Large v3
3. Summarize Qwen3 8B; "Action items only"
4. Save Text followups.md
(Note: "Whisper Large v3" and "Qwen3 8B" are AI models.)
That's an entire AI workflow. You can read it, so you can change it. No developer, no canvas
full of boxes and arrows, no wondering what it did while you weren't looking.
You chain local models together by writing a numbered list, where each row takes one file in
and gives one out. Position is the wiring — by default every row takes whatever the row above
it produced, and you point elsewhere with a number in parens when you need to. Those two rules
give you a full directed acyclic graph written as a plain list you can diff in git or paste
into a chat.
It all runs on your own Mac, so your files stay put and running it costs nothing. Keep using
frontier agents for the thinking and the one-off stuff. When something becomes a weekly habit,
move it here and it becomes a button you press.
MLXUI runs these readable .cat workflows through its Pipeline Runner, so the models you
install in the catalog are the same models your workflows use — no separate setup.
✨ What's New
Chat workspace — give a chat a folder, files and a side panel
Chats can now work on real files. Each conversation gets its own granted folder, a + menu to
attach files (drag-and-drop works too), and a right sidebar with Context (what the model
was told) and Session (what it did). Attachments are read inline within a budget, and images
go through OCR when the model can't see them. Files in this chat lists everything touched,
with previews, a diff for every edit, and a marker on anything still waiting for approval.
One click turns an available file into one that is in this chat.
edit_file tools — the agent can make exact-string edits to files in granted folders (CRLF
and symlink aware). The approval sheet shows precisely what will change before you accept.
Export a chat — Markdown (reasoning folded away, tool calls quoted) or full-fidelity JSON,
from the chat header or the conversation list. Works for every chat model.
Four new models
Qwen3.8 27B — a 27B hybrid model: a 3-bit build (~12.7 GB, suits 18–24 GB Macs)
and an official 4-bit build. Text-only; usable in Browse, chat and AI Workflows.
MiMo V2.6 9B — a 9B model (OptiQ 4/8-bit), text-only, in Browse, chat,
AI Workflows and the Local Server.
Gemma 4 E4B — chat-only for now. The vision build is withheld until the upstream
mlx-swift-lm Gemma 4 vision loader handles the KV-shared layers.
Clef Flash — a 9B decision model on the same seam as Laya and CLM, with a joint
schema head, a native Swift/MLX port of its backbone, and its own Run UI. It joins the
Classify, Gate and Score rows. Decision models now have their own Deciders
section in Browse (Laya 0.4B, CLM 8B, Clef Flash).
AI Workflows — kind rail, smarter row defaults and a basic gallery
Kind rail — an optional strip showing what kind of file flows between rows (text, image,
audio, …). It shows where a row's input comes from, outlines a mismatch, and previews each
row's result as it finishes.
Rows that start a flow now help you. The empty-flow picker is driven by what a row can
accept, new rows get sensible defaults, and warnings name exactly what is missing. Generate
Image / Video / Sound rows have a Use my prompt checkbox, and picking a non-adjacent input
row sets the wiring for you in a single undo step.
Upscale is a real step — with scale= honoured (1–4×) and SeedVR2 3B as its model.
Basic gallery — new beginner-friendly sample flows ship with sample inputs.
Import Flow is on, and models that share a repo now show their own names in row menus.
Sturdier runtime
An MLX error inside a Run window or a chat turn no longer takes the app down — it stops the turn
and says what happened. Served and in-flow models share one memory budget with real-size
accounting, so a load can't be freed while something is still using it, and refusals name who
holds the model.
Local Server — let other apps on your Mac use your models
Turn on Serve on any installed chat model's page and other apps on the same Mac can talk to
it at http://127.0.0.1:1212. Three wire formats share one pipeline, so the tools you already
use work unchanged:
Codex (OpenAI Responses dialect, including previous_response_id)
The Connect your app sheet gives a copy-paste setup block for every client, with plain facts
about the model (usable context on this Mac, whether it can call tools). Settings → Local
Server has the on/off switch, the port, the served models, and a metadata-only request log.
Tool calling for coding agents. Tool calls are returned to the client app — MLXUI never
runs them. Streaming, reasoning (reasoning_content, off unless the request asks) and
cached_tokens usage work across the three dialects.
Prompt cache. Requests that share a prefix (a system prompt, a growing agent
conversation) only prefill the new tail. Snapshots live in RAM only, never on disk, and always
yield memory to model weights.
No request can freeze your Mac. Before prefill, a memory-admission check estimates what a
request needs and either refuses it (context_too_large_for_memory), clamps the reply, or
evicts other idle models first. A prompt past the model's window is a clean 400.
Dedicated server mode. Raise the memory budget, preload and pin every served model so
the first request doesn't wait for a load. A new Local Server screen and Home status card
show what is loaded, what is pinned and how requests are doing. Starting a flow or chat that
would push out a served model asks first.
Loopback only, opt-in. It listens on 127.0.0.1 / [::1] and rejects any request that
isn't addressed to localhost, any Origin, and bodies over 8 MB. It adds no outbound traffic.
Your local models only. Installed MLX chat models, plus Apple Foundation Models where
available. A remote-provider model (Claude, GPT, …) is never served, so your API keys are
never exposed through localhost.
Content is never logged. The request log keeps time, status, model, token counts and
duration — not prompts or replies.
Keeps running with the window closed while it is serving; un-serve everything and the
app quits as usual.
Keychain — fewer prompts, clearer consent
Provider keys and your HuggingFace token now live in the data-protection keychain, so
rebuilding or updating the app no longer re-prompts for access, and an old login-keychain item
is migrated once the first time it is read. Checking whether a key exists never reads the
secret, which ends the burst of Keychain prompts when opening Settings. Settings has
a one-row HuggingFace token (Set Up… / Change… / Remove) and Providers is split into
Search Providers and AI Model Providers, listing only the ones you have configured. Before a
key for a third-party provider is saved you are asked to Agree and Save to a plain sentence
saying what will be sent, to whom, and under whose account; Remove deletes the key and stops it.
CLM 8B — a bigger decision model for long, varied text
CLM 8B is Laya's big sibling: an 8B decision encoder (a frozen Qwen3-8B encoder with two small
heads) that answers typed multiple-choice, scored, or yes/no questions about a block of text, with
a probability per option and a confidence score — and never generates text. Where Laya 0.4B tops
out at 512 tokens, CLM takes 2048 per text with no cap on the number of options, which suits an
invoice with its line items, a contract clause, or an email thread. It has its own Run UI and
joins the Classify, Gate, and Score rows in AI Workflows (Laya stays the default).
About 9 GB peak RAM; English only; Apache-2.0.
Laya 0.4B — a decision model, not a chat model
A new model category. Laya answers typed multiple-choice, scored, or yes/no questions about
a piece of text — ticket/email routing, intent, moderation, "does this need a human?" — in
about 13ms per question, with no generated text and a confidence score on every answer. It's
a from-reference Swift/MLX port of Laya's ModernBERT-large encoder (including the
sliding-window attention the existing ModernBERT embedder doesn't have), plus its decision
head, per-option [MASK] prompt builder, and temperature calibration. Dedicated Browse + Run
UI now; AI Workflows wiring is next.
AI Workflows — output viewer, workspace flows, and human-input polish
A batch of fixes and additions across the Flow surface: a proper output viewer (Quick Look,
"Open in ", Export a Copy), workspace flows that resume automatically after an
install they asked for and can be stopped mid-run, a Flow tab in the row inspector with
capability flags and Reveal in Finder, better Human Input / Ask Human rows (prefilled
defaults, never a blank question), a Frame view for image-region tasks, an honest Row-text
editor with a real Template pattern editor, and a concurrency-safety pass across
FlowKit/Core/Modules.
AI Workflows — automate your models with a visual, numbered flow
A brand-new way to chain your installed models together. Reached from Automate → AI
Workflows, a flow is a simple numbered list — each row picks a task (and a model, if it
needs one) — that runs top to bottom, with live progress dots and cancel. Comes with a
gallery of ready-made sample flows, plus save, open, import and export for your own.
~40 built-in row types — text tools (split, filter, dedupe, template), image tools
(resize, crop, convert, watermark), record/table tools, document retrieval (build and
search an index), OCR and image-description rows, and more
Per-row output caching — re-running a flow only redoes the rows that actually changed
Human-in-the-loop rows — a flow can pause and wait for you to type an answer before
continuing, with an optional timeout
Remote model providers — a row can call Claude, GPT, DeepSeek, or a LAN endpoint using
your own API key (stored in Keychain), or run certain tasks fully on-device via Apple
Intelligence; every remote row is clearly marked before it can send anything off-device
Web Search row — search the web from inside a flow with your own Tavily or Brave key
Workspaces — group related flows in one folder that share files; pair a "build" flow
(indexes your documents) with an "ask" flow (queries them) for a one-card document-Q&A
setup, with a bundled "Ask Your Docs" starter
Missing-file warnings — a flow now tells you up front if a file a row needs isn't
there, instead of failing partway through the run
In-flow file picker — reuse a file already in the flow, add a new one, or create a
folder, instead of a bare system file dialog
A real Settings window — tabs for Models (storage usage, clear cache), Providers,
Tools (every agent capability, with an audit log), and Privacy (a plain-language summary
of what can leave your Mac), plus Local Server
SAM3 — image segmentation
Click points on an image and SAM3 produces a mask you can save as a PNG. Also usable as a
"Segment" step inside AI Workflows.
WAN 2.1 — text-to-video generation (hidden in-app)
Type a prompt, get a short generated video, exported to MP4 with a built-in player.
SeedVR2 — image upscaling (rolling out)
A 3B-parameter upscaling model, added as a new Browse entry.
New models in the catalog
GLM-OCR — a new OCR model, usable in Browse and in AI Workflows
Qwen3.5-9B and Qwen3.5-9B Vision — new chat and vision models
Faster, cheaper installs — if two catalog entries share the same underlying model
files, installing one no longer re-downloads it for the other
MusicGen — type a description, get music back
MusicGen-small is now in the catalog. Type a musical description (e.g. "happy rock with drums"),
pick Short / Medium / Long, and the model generates a WAV file entirely on-device using Apple
Silicon. Playback is inline; you can also Save WAV to keep the file.
The engine is a bespoke Swift/MLX port of the MusicGen autoregressive pipeline: a T5-base text
encoder conditions a 24-layer causal transformer that predicts 4 interleaved EnCodec codebook
streams, which are then decoded by a GPU-accelerated EnCodec decoder back to 32 kHz mono audio.
The whole run — weights, encode, decode loop, audio decode — stays on-device.
Model: jasonvassallo/mlx-musicgen-small
(≈ 2.5 GB install including the bundled 32 kHz EnCodec weights, CC-BY-NC-4.0,
non-commercial use only).
SDXL-Turbo — a second image-generation engine, 4-step on-device diffusion
Stability AI's SDXL-Turbo joins FLUX.1 in the Image Generation category. It's a from-reference
Swift/MLX port of the SDXL UNet, dual CLIP text encoders (CLIP-L + OpenCLIP ViT-G/14), and the
AutoencoderKL VAE, driven by a 4-step DDIM sampler with no classifier-free guidance — a text
prompt becomes a 512×512 image in seconds, entirely on-device. Like Flux, it lives in its own
isolated module folder (Modules/StableDiffusion/) so it can evolve independently of every
other model type.
FLUX.1 Image Generation — text prompt → on-device image, no Python
FLUX.1-Lite-8B (8-bit) is now in the catalog with a full native Run UI. Type a prompt,
hit Generate, and watch a 512×512 image appear — CLIP + T5 text encoding, a native
Flux transformer, and VAE decode all running on Apple Silicon via MLX. No Python
environment, no API keys, no round-trips to the cloud.
The image module follows the same isolated-module pattern as Whisper and Kokoro: the
Flux folder is self-contained and can be updated independently of every other model type.
Cancel Install — mid-download cancel now reliably cleans up
Canceling a model install while it was downloading could leave stale files in the
temp downloads/ folder. That's fixed: canceling now atomically removes the partial
download so disk space is always reclaimed correctly.
Agentic Chat — Local AI that can actually do things
Chat models can now call tools mid-conversation. A single local model can browse the web,
run calculations, transcribe audio, read images, and work with files you grant — all on-device,
all private. Every tool call is shown transparently; anything with side-effects waits for
your explicit approval before it runs.
Tool
What it does
Requires approval
fetch_url
Fetch any webpage and return its text
—
calculator · datetime · run_javascript
Math, dates, and JS evaluation
—
embed_text · summarize · semantic_search
Use your installed embedding and LLM models as tools
—
transcribe_audio
Transcribe an audio URL with any installed Whisper model
—
ocr_image
Extract text from an image URL using any installed OCR model
—
read_file · list_directory · search_files
Browse and read inside folders you explicitly grant
—
write_file · edit_file
Write or exact-string edit files inside granted folders (diff shown)
✅
write_clipboard
Copy text to the clipboard
✅
run_shell(Direct build only)
Run any shell command on your Mac
✅
Ternary-Bonsai-27B — 27B parameters in ~8.4 GB
Prism ML's Ternary-Bonsai-27B — a
2-bit hybrid GatedDeltaNet model — is now in the catalog. Download it in one click and run a full
27B-parameter chat model in roughly 8.4 GB of RAM.
MLXUI uses a module system where each model type lives in its own isolated folder.
Adding a new model type touches only that model's folder — updating Whisper never
touches Kokoro, and a contributor adding image-generation support doesn't need to
understand the chat pipeline.
Modules/
Whisper/ ← one model type, one folder
WhisperModule.swift registers the module with the ModelRegistry
WhisperEngine.swift wraps the MLX inference
WhisperSDK.swift factory that creates the pipeline stage
WhisperUI.swift the SwiftUI Run view
WhisperStage.swift pipeline stage (input/output types for chaining)
The rule: each module may import Core (shared types, protocols) and its own
external package — never another model's module. All cross-model knowledge lives in
Core + the ModelRegistry, which are small and rarely change.
See also: StandAloneRunner.md for per-model file isolation,
storage layout, and engine-to-model dispatch.
Getting Started
Requirements
macOS 14.0 (Sonoma) or later
Apple Silicon (M1 or later)
Xcode 15.0+
Build
git clone https://github.com/your-org/MLXUI.git
cd MLXUI
open PipelineStudio.xcodeproj
Two schemes are available:
Scheme
Distribution
Sandbox
MLXUI
Mac App Store
✅ sandboxed
MLXUI-Direct
Notarized standalone
—
Select your preferred scheme, pick My Mac, and press ⌘R.
First run
The app opens with a catalog of ~28 models organized by category
Browse or search with ⌘K
Click Install on any model — it downloads from Hugging Face
Once installed, click Run to open the model's dedicated interface
Installed models live under ~/Library/Application Support/AI Browser/models/
Contributing
The core contribution path: add a Run UI for a model type that doesn't have one yet.
Every model in the catalog should eventually have a working Run button.
How to add a new model type
Pick an unsupported row from the Engine coverage table
above (the ⬜ rows)
Image generation Run UI (SD 2.1, etc.) — Flux and SDXL-Turbo are now done; other
diffusion architectures still need a prompt → image Run view
More music generation models — MusicGen-small is done; medium and large variants need
catalog entries and engine config updates
Pipeline stages — wire up existing modules into longer chains (e.g. OCR →
summarize)
Model architecture ports — PaddleOCR, DeepSeek-OCR, dots.ocr (see
ModelSupport.swift)
Add catalog entries — pick an architecture from the To be supported
table above. 54 of 68 architectures already have ready-to-download 4-bit quants
(marked ✅) and 2 more have bf16 uploads (marked 🟡). Find the model on
Hugging Face, add it to browser2.json,
and open a PR.
Project conventions
SwiftUI + async/await — no Combine
@Observable macro for state (macOS 14.0 minimum)
4-space indent, PascalCase types, camelCase members
Keep changes scoped — one model type per PR
Build must stay green: open the project in Xcode and use Product → Build (⌘B)
Roadmap
Done
Browse, filter, and compare 435+ models
Hardware-aware RAM/bandwidth filtering
One-click install from Hugging Face with progress tracking
Gated-repo auth flow (Hugging Face token via Keychain)
LLM chat Run UI
ASR Run UI (WhisperKit + MLX Whisper + Voxtral)
TTS Run UI (Kokoro + MLXAudioTTS — Qwen3-TTS, Chatterbox, Orpheus)
VLM Run UI (image Q&A — Gemma, Qwen VL, LFM2-VL)
OCR Run UI (PaddleOCR, DeepSeek-OCR, dots.ocr, olmOCR)
Embeddings Run UI (including ModernBERT standalone)
Agentic chat — 10 local tools (web fetch, compute, MLX model calls, file access, shell)
Ternary-Bonsai-27B — 27B 2-bit hybrid attention model in ~8.4 GB RAM
Dual build targets — App Store (sandboxed) + Direct Distribution (notarized standalone)
Image generation Run UI — FLUX.1-Lite-8B: text prompt → on-device image via native MLX Flux module
Fixed cancel-install mid-download leaving stale files in downloads/
SDXL-Turbo Run UI — a second image-generation engine: SDXL UNet + dual CLIP encoders +
4-step DDIM sampler, native MLX
In progress
Local Server: MCP and served flows, then embeddings/audio endpoints and LAN access
Visual pipeline builder UI
Improved gated-model auth flow
Up for grabs
Image generation Run UI (SD 2.1, etc.)
Additional music generation models (MusicGen-medium, MusicGen-large)
Model comparison benchmark runner
Export pipeline as standalone app
Architecture ports (PaddleOCR, DeepSeek-OCR, dots.ocr — flagged in
ModelSupport.swift)
Populate browser2.json with entries for the 68 unsupported mlx-swift-lm
architectures (54 with existing 4-bit quants, 2 bf16-only, 12 need model conversion).
See the reconciliation table above.
MusicGen Run UI — text description → on-device WAV; T5-base encoder + 24-layer causal
transformer + GPU-accelerated EnCodec decoder, native MLX port
Laya 0.4B Run UI — typed decision/classification questions answered with a confidence
score, no generated text; native MLX ModernBERT-large encoder + decision head port
Local Server — OpenAI-compatible, loopback-only endpoint for your installed chat models
and Apple Foundation Models, with a Connect-your-app sheet and a metadata-only request log
Keychain: data-protection keychain with lazy migration, no secret reads for existence
checks, and explicit consent before a provider key is saved
AI Workflows: output viewer (Quick Look, Open in app, Export a Copy), workspace-flow
auto-resume after install, Flow inspector tab, Human Input/Ask Human polish, Row-text
Template editor, and a concurrency-safety pass across FlowKit/Core/Modules
Local Server: tool calling, Claude Code (/v1/messages) and Codex (/v1/responses)
dialects, prompt cache, memory admission and dedicated server mode
Chat workspace: per-chat folders, attachments with OCR, Context/Session sidebar, file
previews and diffs, edit_file, and Markdown/JSON chat export
Clef Flash decision model, Qwen3.8 27B, MiMo V2.6 9B and Gemma 4 E4B (chat) in the catalog
AI Workflows: kind rail, row-source defaults, Upscale step and a basic sample gallery
