Miles

Enterprise-Grade Reinforcement Learning for Large-Scale Model Post-Training

| Website | Documentation | Quick Start | Supported Models | Miles Diffusion | Blog | Slack (#miles-rl) |

---

News

• [2026/09] 🔥 SGLang and Miles add day-0 support for DeepSeek-V4.1 (blog).
• [2026/09] 🔥 Miles v0.1 technical report is out: Miles v0.1: Production-Level Post-Training.
• [2026/09] Post-training with Miles to understand and generate the multimodal world (blog).
• [2026/08] 🔥 Miles v0.1 is released! Read the blog posts: Announcing Miles v0.1 and Miles v0.1: Production-level Post-training.
• [2026/08] SGLang and Miles add day-0 support for Qwen3.8 (blog).
• [2026/07] Towards Blackwell-Native 8-bit and 4-bit RL: End-to-End MXFP8 and NVFP4 RL in Miles (blog).
• [2026/07] 🔥 SGLang and Miles add day-0 support for Kimi K3 (blog).
• [2026/07] On-policy distillation lands in Miles (blog).
• [2026/07] 🔥 SGLang and Miles add day-0 support for Inkling, a frontier multimodal model (blog).
• [2026/07] DeepSeek-V4 Flash RL training comes to AMD Instinct MI355X with Miles (blog).
• [2026/06] Miles: a PyTorch-native stack for large-scale LLM RL post-training (PyTorch blog).
• [2026/06] SGLang and Miles add day-0 support for NVIDIA Nemotron 3 Ultra (blog).
• [2026/05] No token left behind: token-in-token-out in Miles (blog).
• [2026/04] Updating 1 T parameters in seconds: P2P weight transfer in large-scale distributed RL (blog).
• [2026/04] 🔥 DeepSeek-V4 on day 0: from fast inference to verified RL with SGLang and Miles (blog).

About

Miles is a high-performance, enterprise-ready reinforcement learning framework for large-scale model post-training. It pairs SGLang for high-throughput rollout with Megatron-LM for scalable training, and ships the precision, stability, and observability features an RL run needs at trillion-parameter scale. FSDP2 and torchtitan backends are also supported. See Training Backends.

"A journey of a thousand miles begins with a single rollout."

Performance

• Fully async RL. Rollout and training workers are decoupled, with configurable on- and off-policy schedules, a pipeline tuned for fewer bubbles, and customizable async rollout and eval modes. See Fully Async RL.
• Fast agentic rollout. Generation runs on SGLang behind a router that spreads requests across engines, preserves per-request metadata, and health-checks the fleet. Tuned for multi-turn agentic workloads.
• Fast weight updates. New weights reach the engines in-loop in seconds, even on a trillion-parameter model such as Kimi-K2.6, with P2P RDMA as the fast path for disaggregated setups.
• Low-precision training. MXFP8 and NVFP4 training with a numerically stable RL recipe that reduces precision-induced divergence. FP8, INT4 QAT, BF16, and FP16 are also supported.
• LoRA and multi-LoRA. Low-rank adapters train frontier-scale models on a fraction of the GPUs, and the same adapters load straight into SGLang for rollout.

This is the beginning of the README. Read the rest at the original. Read the full README ↗