vllm-metal

Architecture

Upstream vLLM supplies the API server, scheduler, and paged block manager; mlx_lm supplies the token-wise model layers; vllm-metal owns the request-aware attention path — the paged varlen kernel, M5 NAX prefill, and speculative decoding.

Requirements

• macOS 15 (Sequoia) or later, on Apple Silicon

Supported Models

vllm-metal supports a growing set of models on Apple Silicon. See the full matrix in docs/supported_models.md.

Installation

• Stable release: Install with Homebrew, then run vllm without activating an environment.

  brew tap vllm-project/vllm-metal https://github.com/vllm-project/vllm-metal
  brew install vllm-project/vllm-metal/vllm-metal
  

• Latest development build: Use the curl installer, then activate its environment. No compiler needed.

  curl -fsSL https://raw.githubusercontent.com/vllm-project/vllm-metal/main/install.sh | bash
  source ~/.venv-vllm-metal/bin/activate
  

• Contributing Python or kernel code: Follow the source setup for an editable checkout and native build tools.