Splash

A local inference engine for Apple silicon, built around the model.

Splash runs coding agents and OpenAI or Anthropic compatible applications on one Mac. It combines DFlash 2 speculative decoding, specialized Metal kernels, and automatic memory planning, with vision, tool calling, and a built-in chat page. It reuses cached prefixes and batches concurrent requests automatically.

Quick start

Download the Splash desktop app for an Apple M3 or newer on macOS 26.4 or later, open the disk image and drag Splash into Applications.

The command line needs the same Mac and Homebrew:

brew install incoai/tap/splash
splash serve --model unsloth/Qwen3.8-27B-GGUF:UD-Q4_K_M

The first run downloads the model and its matching draft and starts serving on 127.0.0.1:8000. Later starts reuse the downloads; leave room on disk for them (storage requirements).

Once it prints Ready, leave this terminal open. Open http://127.0.0.1:8000 in your browser, or run an installed coding agent from another terminal:

splash opencode    # or: splash claude / splash codex / splash hermes / splash pi

For a server started with another --port, pass the same --port to the agent.

Press Ctrl+C in the server terminal to stop Splash. For LM Studio Bionic, follow its Splash setup guide.

Use the API

OpenAI Chat Completions, Responses and Completions, and Anthropic Messages, with streaming, tool calls, JSON Schema output, and base64 images and PDFs:

curl http://127.0.0.1:8000/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "unsloth/Qwen3.8-27B-GGUF:UD-Q4_K_M",
    "messages": [{"role": "user", "content": "Explain speculative decoding in one sentence."}]
  }'

Reasoning follows the model default; "reasoning_effort": "none" turns it off in Chat. Anthropic Messages requests reason only when they set thinking. Reasoning settings · Tool calls · API details

Models

Splash supports these model families, with a matching DFlash2 draft selected automatically:

Model	GGUF example	MLX 4-bit
Qwen3.8-27B	unsloth/Qwen3.8-27B-GGUF:UD-Q4_K_M	mlx-community/Qwen3.8-27B-4bit
Qwen3.6-35B-A3B	unsloth/Qwen3.6-35B-A3B-GGUF:UD-Q4_K_M	mlx-community/Qwen3.6-35B-A3B-4bit
Ornith-1.5-122B	unsloth/Ornith-1.5-122B-GGUF:UD-Q4_K_XL	ornith-ai/Ornith-1.5-122B-MLX-4bit
Qwen3.8-Flash-Next	ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF:Q2_0	—
Flint	unsloth/flint-GGUF:UD-Q4_K_XL	—

Unsloth GGUF variants span 1–8 bits, including mixed-precision UD formats up to UD-Q8_K_XL, whose BF16 tensors load too. Whole-BF16 and F16 models are not supported: the installer refuses a GGUF whose projections and experts are all BF16, F16 or F32. Prism ML Ternary Bonsai 2 is also supported in PQ2_0 (7.2 GB), including vision. Pass OWNER/REPO:VARIANT to --model, as in the quick start. MLX targets can be affine 2–8-bit, including mixed precision, mxfp4 or nvfp4. NVIDIA's and unsloth's NVFP4 checkpoints and NVFP4 GGUFs also load. Smaller variants run on 24 GB Macs. 27B variants · 35B variants · Ornith-1.5-122B variants · Flash-Next variants

Vision and the tokenizer come from the target model's source. Model loading and compatibility · Supported formats

To serve files already on this Mac, pass the target's GGUF file or GGUF/MLX directory directly, and a compatible local DFlash2 draft:

splash serve --model local/qwen \
  --model-dir /models/Qwen3.8-27B-GGUF/Qwen3.8-27B-UD-Q4_K_M.gguf \
  --draft-model /models/Qwen3.8-27B-DFlash2

This prepares local files without contacting the Hub; --model is the name reported by the API. Omit --draft-model to use the family's draft from the Hub cache, or add --mmproj /models/mmproj-BF16.gguf for a GGUF's vision. See local model files for details.

Settings

Memory and context are sized automatically, up to the model's native context window. To set your own limits or cache options, add these to splash serve:

Option	Purpose
--max-memory 28G	Cap Metal memory use.
--idle-release off	Keep the model in memory while idle (default: release after 10m).
--max-context 100K	Set the context limit. Up to 1M tokens.
--language-only	Skip vision; serve text only.
--kv-format bf16	Use BF16 KV cache. Default: 8-bit (INT8).
--state-format bf16	Use BF16 recurrent states, half the size. Default: FP32.
--activations bf16	Use BF16 activations. Default: auto.
--disable-ane	Prefill on the GPU alone. Default: the 27B also uses the Neural Engine.
--max-cache-disk 16G	Cap the SSD cache (default: auto, up to 32 GiB; 0 turns it off).
--persistent-cache	Keep the SSD cache across restarts. Off by default.

This is the beginning of the README. Read the rest at the original. Read the full README ↗