exo: Run frontier AI locally. Maintained by exo labs.
exo connects all your devices into an AI cluster. Not only does exo enable running models larger than would fit on a single device, but with day-0 support for RDMA over Thunderbolt, makes models run faster as you add more devices.
Features
- Automatic Device Discovery: Devices running exo automatically discover each other - no manual configuration.
- RDMA over Thunderbolt: exo ships with day-0 support for RDMA over Thunderbolt 5, enabling 99% reduction in latency between devices.
- Topology-Aware Auto Parallel: exo figures out the best way to split your model across all available devices based on a realtime view of your device topology. It takes into account device resources and network latency/bandwidth between each link.
- Tensor Parallelism: exo supports sharding models, for up to 1.8x speedup on 2 devices and 3.2x speedup on 4 devices.
- MLX Support: exo uses MLX as an inference backend and MLX distributed for distributed communication.
- Multiple API Compatibility: Compatible with OpenAI Chat Completions API, Claude Messages API, OpenAI Responses API, and Ollama API - use your existing tools and clients.
- Custom Model Support: Load custom models from HuggingFace hub to expand the range of available models.
Dashboard
exo includes a built-in dashboard for managing your cluster and chatting with models.
Benchmarks
Quick Start
Devices running exo find each other automatically, without any manual configuration, when they are on the same network and run the same version of exo (see Adding another Mac). Each device provides an API and a dashboard for interacting with your cluster (runs at http://localhost:52415).
You can run exo with the macOS app, the easiest way on a Mac, or from source on macOS or Linux.
macOS App
exo ships a macOS app that runs in the background, in your Mac's menu bar.
The app needs a Mac with Apple silicon and macOS Sequoia 15.1 or later. RDMA over Thunderbolt needs macOS Tahoe 26.2 or later.
Download the latest version: EXO-latest.dmg. Open it, drag EXO to your Applications folder, and open EXO from there. The first time, macOS asks whether you want to open an app downloaded from the internet: click Open.
You can also install it with Homebrew:
brew install --cask exo
What EXO asks for, and why
-
Local Network access. macOS asks whether EXO may find devices on your local network. EXO needs this to find your other devices running exo and share the work with them. If you said no, turn EXO on in System Settings → Privacy & Security → Local Network, then quit and reopen EXO.
-
Thunderbolt network setup (optional, needs your administrator password). EXO offers to set up networking for Thunderbolt in its menu when Macs are connected with Thunderbolt cables; you can also do it in Settings › Cluster. Macs that share a Wi-Fi or Ethernet network don't need it. If you agree, EXO installs a small system service (io.exo.networksetup) that:
- creates a network location called
exo and switches this Mac to it;
- gives each Thunderbolt port its own network service, which gets an address automatically;
- turns off Thunderbolt Bridge, which can flood the network with packets when three or more Macs are cabled in a loop.
The service runs when your Mac starts and about every 30 minutes, to keep these settings in place. The exo location starts with default settings, so anything you changed in your usual location (such as a fixed IP address, DNS servers or proxies) doesn't apply while it's in use. To undo the setup, uninstall EXO. To undo it but keep EXO, run the uninstall script described there: it restores your network settings and keeps your downloaded models, and EXO asks again before it sets up the network.
-
Opening at login. EXO's welcome window has Open EXO at login ticked, so your Mac is ready to work with the others after a restart. Untick it there, or change it later in EXO's Settings › General or in System Settings → General → Login Items & Extensions.
-
Notifications. EXO doesn't ask at launch. When an update is ready to install, it leaves a quiet notification in Notification Center; you can turn these off in System Settings → Notifications. You can always check for updates in EXO's menu or Settings.
Adding another Mac
- Install EXO on the other Mac and open it.
- Put both Macs on the same local network (Wi-Fi or Ethernet), or connect them with a Thunderbolt cable.
- Make sure both Macs run the same version of exo. Devices running different versions don't join each other.
The new Mac shows up in the dashboard within a few seconds.
To see which version a Mac runs:
- in the app: choose About EXO from the menu bar icon, or open Settings › General;
- from source: run
uv run exo --version.
The app and a source checkout number their versions differently. The app uses its release number (such as 1.0.71), while a source checkout uses the version in pyproject.toml (such as 0.3.70). Because the default cluster name is the version, an app and a source checkout don't join each other by default. To mix them, give every device the same cluster name.
If a Mac doesn't show up:
以上为 README 的开头部分。其余内容请阅读原文。 查看完整 README ↗
