vLLM Metal is a plugin that enables vLLM to run on Apple Silicon Macs using MLX as the primary compute backend. It unifies MLX and PyTorch under a single lowering path.
Latest News 🔥
- [2026/10] Experimental Recursive Flint INT4 support is now available on the MLX backend, including concurrent serving.
- [2026/10] DiffusionGemma now supports Jev-like System 1 decisions on Metal, with median decision latency under 0.2 s on an M5 Pro. See the structured decisions guide.
- [2026/10] Desktop inference competes with your other applications for memory. Sleep mode lets an idle model server return memory to those apps without shutting down. Read Abhigith Neil Abraham's walkthrough.
- [2026/10] Run local models with vLLM Metal through llmman. See the setup recipe.
- [2026/09] SiliconBench, which evaluates vLLM Metal and other serving engines on speed, memory use, and output fidelity, was accepted at NeurIPS 2026's Evaluations and Datasets Track.
- [2026/09] Read our official vLLM announcement, covering concurrent serving on Apple Silicon and v0.28.0.
- [2026/08] vLLM Metal now uses M5 NAX tensor units to accelerate MHA, GQA, and MQA prefill.
- [2026/08] Qwen3.8 now runs on Metal!
mlx-community/Qwen3.8-27B-8bit serves a 27B hybrid SDPA + GDN linear model on a single Apple Silicon Mac.
- [2026/04] We released the new version v0.2.0! Unified paged varlen Metal kernel is now the default attention backend. 83x TTFT, 3.6x throughput compared to v0.1.0.
Architecture
Upstream vLLM supplies the API server, scheduler, and paged block manager; mlx_lm supplies the token-wise model layers; vllm-metal owns the request-aware attention path — the paged varlen kernel, M5 NAX prefill, and speculative decoding.
Requirements
- macOS 15 (Sequoia) or later, on Apple Silicon
Supported Models
vllm-metal supports a growing set of models on Apple Silicon. See the full matrix in docs/supported_models.md.
Installation
-
Stable release: Install with Homebrew, then run vllm without activating an environment.
brew tap vllm-project/vllm-metal https://github.com/vllm-project/vllm-metal
brew install vllm-project/vllm-metal/vllm-metal
-
Latest development build: Use the curl installer, then activate its environment. No compiler needed.
curl -fsSL https://raw.githubusercontent.com/vllm-project/vllm-metal/main/install.sh | bash
source ~/.venv-vllm-metal/bin/activate
-
Contributing Python or kernel code: Follow the source setup for an editable checkout and native build tools.
