Evolve LLM

Run open language models on your own GPU from a web page, including models far larger than your GPU's memory. Evolve LLM is an inference engine written in Rust. It is compiled to WebAssembly and computes every token with WebGPU compute shaders. Open the page, pick a model, and chat with it. There is nothing to install, no account, and no server that sees your messages.

Try it: evolvetech.dev/play/evolve-llm

Status: early release. The engine runs dense models (Llama, Mistral, Qwen2, Qwen3, SmolLM) and mixture-of-experts models (Qwen3-MoE, Mixtral, OLMoE, DeepSeek V2 and V3, Kimi K2) from GGUF files, single or split. It is tested in Chrome on Linux with NVIDIA and Intel GPUs and against reference implementations (see Verification). The largest models in the catalog have not been run end to end yet; see Limits.

Browser workspace and Pi

Ask the main chat to build an app or game, or open Terminal for a persistent WASM text workspace with pipes and file redirects. Enter demo snake to play the included example, or preview game.html to run your own single-file HTML app in an isolated frame. Pi can test apps, inspect runtime errors and edit them. Load a tool-capable model, then enter pi <task> to run Pi agent core with sandbox file and research tools. Web access is explicit; there is no host shell or server inference. See commands, limits, permissions and self-hosting.

What it does

• Downloads a model once. Choose from the catalog, browse any Hugging Face repository, or paste a GGUF file's path. Files stream into the browser's private storage and resume after a dropped connection; split models download part by part. You can also open GGUF files already on your device; they are read in place.
• Runs it on your GPU. Weights upload in 16 MB slices, so loading never needs the whole file in memory.
• Runs models that do not fit. Give the model a GPU memory budget in Settings. What fits stays on the GPU; the rest is read from storage while the model runs. Mixture-of-experts models keep their attention weights on the GPU and read only the experts each token chooses, through an expert cache. The operating system's file cache keeps recently read weights in RAM.
• Searches the web when you let it. With Internet access on, models whose chat templates take tools can call web_search and open_url; other models get search results with each question. Searches go from the browser to Wikipedia and DuckDuckGo, and pages are read through Jina Reader. Each search and page shows above the answer with its sources.
• Chats with the model's own template. Prompts are formatted by the Jinja chat template in the GGUF file. Each new turn only computes the part of the conversation that is not already in the KV cache.
• Shows its reasoning. Qwen3 thinking is shown in a collapsible block and can be turned off.

With web access off, nothing leaves the device except model downloads from Hugging Face. There are no analytics.

Supported models

	Supported
Dense architectures	llama (Llama 2 and 3, Mistral, SmolLM2, TinyLlama), qwen2 (Qwen2 and 2.5, Coder), qwen3
Mixture of experts	qwen3moe (Qwen3 30B-A3B to Coder 480B-A35B), llama with experts (Mixtral), olmoe, deepseek2 (DeepSeek V2, V3, V3.1 and R1, Kimi K2, Moonlight)
Attention	Grouped-query attention, multi-head latent attention (DeepSeek), YaRN and linear position scaling, Llama 3 rope factors
Weight formats	IQ1_S, IQ1_M, IQ2_XXS, IQ2_XS, IQ2_S, IQ3_XXS, IQ3_S, IQ4_NL, IQ4_XS, TQ1_0, TQ2_0, MXFP4, Q2_K, Q3_K, Q4_K, Q5_K, Q6_K, Q4_0, Q4_1, Q5_0, Q5_1, Q8_0, F16, BF16, F32
Tokenizers	Byte-level BPE (GPT-2, Llama 3, Qwen2, SmolLM, DeepSeek V3 and Kimi K2 pre-tokenizers) and SentencePiece-style BPE
Files	Single GGUF files and split sets (name-00001-of-00004.gguf)
Not yet	Gemma, Phi, gpt-oss, hybrid linear-attention models (Qwen3.5 and 3.6, Kimi K3, DeepSeek V4, GLM-5), fused gate-up expert tensors, images

Browser support

WebGPU is required: Chrome or Edge 113 or later on Windows, macOS, ChromeOS and Android; Safari 26 or later; Firefox 141 or later on Windows. On Linux, Chrome needs chrome://flags/#enable-unsafe-webgpu and Vulkan enabled. The page explains what to do when WebGPU is missing. Large models also need browser storage: Chrome lets a site use a large share of free disk space.

How it works

The Rust crate owns everything that touches the model: GGUF parsing, weight layouts, the WGSL kernels, the placement of weights between GPU and storage, tokenizers, chat templates, the KV cache and sampling. JavaScript handles the page, downloads, file reads and web tools. The same engine runs natively through wgpu on Vulkan, Metal or Direct3D 12, which is how it is tested. See docs/architecture.md for the kernels, layouts and streaming.

src/gguf.rs        GGUF header parsing from a file prefix, split-file merging
src/quant.rs       Dequantization and the GPU layouts of every weight format
src/iq_tables.rs   IQ code books from ggml (MIT, see NOTICE)
src/model.rs       Architecture configuration and tensor names
src/tokenizer.rs   BPE and SentencePiece tokenizers, streaming UTF-8 decoding
src/chat.rs        Chat templates through MiniJinja, with tools
src/gpu/           wgpu device, WGSL kernels, the forward pass and weight streaming
src/source.rs      Where streamed weights are read from
src/session.rs     Prompt-prefix reuse across turns and sampling
src/cpu.rs         A slow f32 reference forward pass for tests
src/web.rs         WebAssembly bindings
web/               The page, the worker, the model catalog, web tools and the Markdown renderer

여기까지가 README 앞부분입니다. 나머지는 원문에서 읽을 수 있습니다. README 전문 보기 ↗