FastFlowLM

⚡ FastFlowLM (FLM) — Unlock Ryzen™ AI NPUs

Run large language models — now with Vision, Audio, Embedding and MoE support — on AMD Ryzen™ AI NPUs in minutes.
No GPU required. Faster and over 10× more power-efficient. Supports context lengths up to 256k tokens. Ultra-Lightweight (17 MB). Installs within 20 seconds.

📦 The only out-of-box, NPU-first runtime built exclusively for Ryzen™ AI.
🤝 A familiar single-command CLI — deeply optimized for NPUs.
✨ From Idle Silicon to Instant Power — FastFlowLM Makes Ryzen™ AI Shine.

FastFlowLM (FLM) supports all Ryzen™ AI Series chips with XDNA2 NPUs (Strix, Strix Halo, Kraken, and Gorgon Point).

---

🔗 Quick Links

🔽 Download | 📊 Benchmarks | 📦 Model List

🐧 Linux Getting Started Guide

📖 Docs | 📺 Demos | 💬 Discord

---

🚀 Quick Start

A packaged FLM Windows installer is available here: flm-setup.msi. For more details, see the release notes.

📺 Watch the quick start video (Windows)

[!IMPORTANT]
⚠️ Use the latest AMD NPU driver — 32.0.203.311 or above (check via Task Manager→Performance→NPU or Device Manager). Earlier versions are no longer supported.
⚙️ Tip:

• RECOMMENDED: Try running Windows Update or Driver Download.
• Official AMD Install Doc (AMD account required).
• Unofficial forum downloads (CAUTION: third-party content not verified by AMD; download and use at your own risk).

After installation, open PowerShell (Win + X → I). To run a model in terminal (CLI Mode):

flm run llama3.2:1b

Notes:

• Internet access to HuggingFace is required to download the optimized model kernels.
• Sometimes downloads from HuggingFace may get corrupted. If this happens, run flm pull <model_tag> --force (e.g. flm pull llama3.2:1b --force) to re-download and fix them.
• By default, models are stored in:
  • Windows: C:\Users\<USER>\.flm\models\
  • Linux: ~/.config/flm/
• During installation on Windows, you can select a different base folder (e.g., if you choose C:\Users\<USER>\flm, models will be saved under C:\Users\<USER>\flm\models\).
• On Linux, you can override the default location by setting the FLM_MODEL_PATH environment variable.
• To disable the startup version check, set FLM_DISABLE_UPDATE_CHECK=1.
• ⚠️ If HuggingFace is not accessible in your region, manually download the model (check this issue) and place it in the chosen directory.

🎉🚀 FastFlowLM (FLM) is ready — your NPU is unlocked and you can start chatting with models right away!

Open Task Manager (Ctrl + Shift + Esc). Go to the Performance tab → click NPU to monitor usage.

⚡ Quick Tips:

• Use /verbose during a session to turn on performance reporting (toggle off with /verbose again).
• Type /bye to exit a conversation.
• Run flm list in PowerShell to show all available models.

To start the local server (Server Mode):

flm serve llama3.2:1b

The model tag (e.g., llama3.2:1b) sets the initial model, which is optional. If another model is requested, FastFlowLM will automatically switch to it. The local server runs on port 52625 (default).

---

📰 In the News

• 08/11/2026 🎉 FLM is now part of ROCm (v1.0.0) — the repo has moved to AMD's open-source ROCm organization.

• 08/11/2026 🎉 FLM releases its first SmolVLA model (v1.0.0) — a Vision-Language-Action robotics policy running on the NPU. See the model card and benchmarks.

• 07/17/2026 🎉 FLM is now part of AMD news. Read our story — from a 2025 university project to AMD.

• 03/11/2026 🎉 FLM now supports Linux 🐧 ! To get started, check out the quick start guide or the Lemonade Server docs, and watch the short video for a quick walkthrough of FLM on Linux via Lemonade 🍋.

• 10/01/2025 🎉 FLM was integrated into AMD's Lemonade Server 🍋. Watch this short demo about using FLM in Lemonade.

---

🧠 Local AI on NPU

FLM makes it easy to run cutting-edge LLMs (and now VLMs) locally with:

• ⚡ Fast and low power
• 🧰 Simple CLI and API (REST and OpenAI API)
• 🔐 Fully private and offline

No model rewrites, no tuning — it just works.

---

✅ Highlights

• Runs fully on AMD Ryzen™ AI NPU — no GPU or CPU load
• Lightweight runtime (17 MB) — installs within 20 seconds, easy to integrate
• Developer-first flow — a familiar single-command CLI, optimized for NPU
• Support for long context windows — up to 256k tokens (e.g., Qwen3-4B-Thinking-2507)
• No low-level tuning required — You focus on your app, we handle the rest

---

📄 License

• All orchestration code and CLI tools are open-source under the MIT License.
• The NPU-accelerated binary kernels are not open source, but are completely free for any use, including non-commercial and commercial use, and may be freely redistributed in unmodified form under the FastFlowLM NPU Binary License.
• Please acknowledge FastFlowLM in your README/project page (or product) as follows:

  Powered by [FastFlowLM](https://github.com/ROCm/FastFlowLM)
  

---

💬 Have feedback/issues or want early access to our new releases? Open an issue or Join our Discord community

---

🙏 Acknowledgements

• Powered by the advanced AMD Ryzen™ AI NPU architecture
• Inspired by the widely adopted llama.cpp and Ollama
• Tokenization accelerated with MLC-ai/tokenizers-cpp
• Chat formatting via Google/minja
• Constrained decoding (tool calls, structured output) powered by MLC-ai/xgrammar
• Low-level kernels optimized using the powerful IRON+AIE-MLIR

---

🛠️ Building from Source

For developers who want to build FastFlowLM from source, we provide CMake presets for a convenient and consistent build experience.

Prerequisites

• Git
• CMake (version 3.22 or higher)
• A C++20 compatible compiler (e.g., GCC, Clang, MSVC)
• Ninja (recommended)

Build Instructions

More details on the exact procedure, with dependencies to be installed, for Linux can be found in linux-getting-started.md.

1. Clone the repository:

   git clone --recursive https://github.com/ROCm/FastFlowLM.git
   cd FastFlowLM/src
   

2. Configure CMake using presets:

   • For Linux:

     cmake --preset linux-default
     

     This will configure the build to install to /opt/fastflowlm.

     To build the Gemma 4 engine from source instead of using the prebuilt engine:

     cmake --preset linux-default -DFLM_BUILD_GEMMA4E=ON
     cmake --build build
     

     CMake caches this option. To switch back to the prebuilt engine, reconfigure with -DFLM_BUILD_GEMMA4E=OFF or use a fresh build directory.

   • For Windows (in a developer command prompt):

     cmake --preset windows-default
     

     Add -DFLM_BUILD_GEMMA4E=ON to build the Gemma 4 engine from source.

3. Build the project:

   cmake --build build
   

4. Install the project (optional):

   • For Linux:

     sudo cmake --install build
     

   • For Windows (with administrator privileges):

     cmake --install build