1bit engine
Documentation: 1bit.gg · Measured results: wiki · Community: Discord
Site: github.com/1bit-MONSTER/site (builds 1bit.gg from this repository's README.md and docs/)
The 1bit engine is AMD's HRX runtime, our own HIP kernels on the Strix Halo GPU, and our NPU
engine, all inside Lemonade. On a Ryzen AI Max+ 395,
every graph node of every model and mode we test runs on the GPU: 43 of 43 configurations in the
census, with 0 nodes on the CPU (#476,
#480,
#481). Qwen3.8-27B (UD-Q4_K_XL) with MTP
speculative decoding decodes at 36.0 / 26.8 / 29.9 tok/s on code / prose / short chats
(#487).
Where it stands on 23 October 2026 (candidate v2026.43; numbers carry their standard and their
power state — lemonade bench standard is Lemonade's own CI benchmark, ours is our sampler/batched
standard, and everything measured after the 23 Oct 01:38 rebuild is 90 W cap, PCM pad (23 Oct)):
| what | number | standard |
|---|
| Qwen3.8-27B UD-Q4_K_XL with MTP, code / prose / short chat | 36.0 / 26.8 / 29.9 tok/s | lemonade bench standard |
| Qwen3-Coder-30B-A3B, single stream | 84.2 tok/s (85.9 in the lemonade-bench cells) | ours / lemonade standard |
| batched, aggregate at 8 concurrent 256-token chats: 27B / Coder-30B / Qwen3.5-4B | 50.7 / 138.1 / 194.3 tok/s | ours, 90 W cap, PCM pad (23 Oct) |
| NPU, 0.6B gate phase | 90.33 tok/s | ours (gate) |
| census on HRX: configurations with 0 nodes off the GPU | 43 of 43 | ours (gate) |
| Lemonade catalog: HRX entries proven on the box | 22 of 22 (43.4–330.7 tok/s chat-short) | lemonade bench standard |
The engine is in the release candidate now; the tag and the packages are the owner's
(docs/releases.md).
Lemonade stays the server you talk to (its catalog, downloads, router and UI). For the models it
hands to 1bit, it runs the engine as one of its backends, behind an OpenAI-compatible API:
- HRX on the Radeon iGPU (docs/hrx.md): AMD's ggml-hrx with its Loom kernels
and our HIP kernels, the engine's one GPU route. For a GGUF, Lemonade starts the engine's
llama-server directly, one HTTP hop
(#475,
docs/serve.md). Placement is strict by
default: the whole graph runs on HRX, or the load stops with the node that would not
(#482,
docs/serve.md). --device cpu runs the
same build on the CPU.
- The NPU engine on the XDNA 2 NPU, in progress: correctness first, speed next
(docs/npu.md).
- Hybrid: the prompt on the NPU engine, the tokens on HRX.
1bit serve --device hybrid hands
the prompt's KV cache from a prefill provider to the HRX llama-server
(docs/hybrid.md).
- DwarfStar, for DeepSeek V4 Flash, GLM 5.x and Qwen3.8-Flash-Next in its own GGUFs, with SSD
expert streaming (docs/dwarfstar.md).
- Laya, opt-in, which picks where each request runs (docs/laya.md), and
the Project Manager, ZAYA1-8B delegating to expert models (docs/pm.md).
- Every Hugging Face model architecture, mapped to a backend by a registry and kept current
by a daily census ().
Vulkan, ROCm, CUDA, ZINC, ComfyUI, Metal/MLX and Windows were cut from the engine's core
(#306) and are not routes of the engine.
Lemonade serves them with its own backends and builds (docs/lemonade.md).
Checked on the hardware. Every HRX pin change and every PR labelled hw-gate passes the
hardware gate on Strix Halo before merge: it builds our HIP kernels into the server as the release
does, runs kernel tests, known answers, KLD against the release pin and long-prompt needles, and
checks that every graph of every gate model is on the GPU
(#478,
#484, docs/hw-gate.md). The
runtime keeps one memory budget for every buffer, and after a GPU fault the server refuses new work
and exits cleanly, so Lemonade can restart the model
(#486). Serve numbers in these docs come from
tools/bench.py, which A/B-measures configurations in the same run (docs/bench.md).
The box's kernel is ours too: Linux v7.3-rc4 plus the IOMMU PerfOpt backports, built from our
config, MOK-signed for Secure Boot, and gated like a pin bump. Moving it to the next upstream tag is
a written step list (docs/kernel.md).
Install: through Lemonade. The packages are Lemonade with the onebit recipe and the engine
(unpack it and run lemond-onebit), and 1bit OS, a Lemonade appliance on a bootable USB image.
On Ubuntu 26.04 the stock lemond from the Lemonade PPA runs the engine through a systemd drop-in,
with no patch to Lemonade (#483,
docs/lemonade.md). The engine has no standalone
package of its own. The first release goes out when every item on the release checklist is done
(docs/releases.md).
Direction: HRX with our HIP kernels, plus the NPU engine, embedded into Lemonade rather than
Lemonade embedded into the engine.1 HRX met the decode gates in
RFC #213, which removed the engine's
Vulkan and ROCm builds. HRX is pinned to AMD's hrx-system and ggml-hrx with our commits on top,
kept current by bump-hrx (docs/hrx.md). ZAYA1-8B
(Zyphra) runs from our llama.cpp on HRX at about 90 tok/s decode in Q4_K_M, matching
transformers. Zyphra's Zamba, Zamba2 and BlackMamba ran only on the removed Vulkan build and are
to be ported to HRX. The working engine is being ported from our private development repository;
see docs/PORTING.md. Measured results are on the
wiki. This repository holds the verified code
without the development history.
License
The source code in this repository is Apache-2.0. See LICENSE.
The release binaries (the Linux package and 1bit OS) also contain two proprietary components,
compiled in: the 1bit NPU runtime and the 1bit HIP kernels for HRX. They are not in this
repository, their source is not published, and they are not under Apache-2.0: they ship under the
1bit Binary License (free to use for any purpose and to pass on unmodified;
third-party parts in NOTICE-BINARY). A build from this
repository without them has no NPU engine and runs HRX on the public kernels only
(NOTICE, docs/releases.md).
How to credit
Credit is welcome and, for the code, required: the Apache-2.0 license keeps the copyright
notices and the NOTICE file with every copy. Three lines are enough:
1bit engine - https://github.com/1bit-MONSTER/engine - Copyright 2026 bong-water-water-bong - Apache-2.0
CITATION.cff carries the same in citation form (GitHub shows it under
"Cite this repository"). If you build on a specific piece, name the file or the pull request it
came from; the history is public.
Thank you
Osmantic / ODS comes first. ODS introduced me to
vibecoding, and that is where all of this started. Without it, this engine would not exist.
The Lemonade team and AMD's developers. Lemonade
is where this engine runs. AMD's developers left breadcrumbs all over the place: the XDNA driver
and XRT, IRON and Peano, HRX, their tested llama.cpp integration, their issues, their examples.
This engine is what following those breadcrumbs built.
The repositories this engine is built on, in order of importance:
| # | Repository | What it gives this engine | License |
|---|
| 1 | amd/xdna-driver | The XDNA 2 NPU kernel driver (amdxdna) the NPU engine runs on | GPL-2.0 (driver) |
| 2 | Xilinx/XRT | AMD's NPU runtime and its API, where the engine's NPU work started | Apache-2.0 (userspace) |
| 3 | Xilinx/mlir-aie | IRON and aiecc: how our own NPU kernels are written and compiled | Apache-2.0 WITH LLVM-exception |
| 4 | Xilinx/llvm-aie | Peano, the C++ compiler for the NPU's AI Engine cores | Apache-2.0 WITH LLVM-exception |
| 5 | ggml-org/llama.cpp | GGUF inference on the Radeon iGPU through HRX (AMD's tested pair), and on the CPU | MIT |
| 6 | ROCm/hrx-system (our fork 1bit-MONSTER/hrx-system carries two commits) | HRX, AMD's HIP Runtime Extended, behind the HRX0 device | Apache-2.0 |
| 7 | torvalds/linux | The kernel, with amdxdna and amdgpu in-tree | GPL-2.0 WITH Linux-syscall-note |
| 8 | huggingface/tokenizers | Every model's tokenizer.json, byte-exact, behind our C ABI | Apache-2.0 |
| 9 |
Also built on nlohmann/json (MIT)
and yhirose/cpp-httplib (MIT).
Every third-party copyright and license is listed in NOTICE. Each project keeps its
own license; nothing here relicenses anyone's work.
