English | 中文
Reef is the first open-source infrastructure for continual self-improving agents.
It connects agent inference, feedback, learning, and versioned delivery. Use it
to train model weights with Slime and SGLang, or improve an agent's harness, including its prompts, rules, and skills.
🚀 Get started |
🗺️ Roadmap |
🗓️ Office Hours |
📣 Launch post |
💬 Join Discord |
📱 Join WeChat Group
📰 News
- 2026-11: We are refactoring Reef to a control plane + workers architecture. Some recipes and examples can fail on
main until the refactor is complete. If a recipe fails, please open an issue.
- 2026-11: A scenario can split its traffic across its served artifacts #1227, also through Semantic Router #859. The new
python harness adapter evolves a Python program #1242.
- 2026-10: Tinker runs as a training component #901. Episodes and agents can run in Modal Sandboxes #942. The artifact repository can live on Hugging Face #1003. New recipe: OPD #683.
- 2026-09:
reef-infra v0.1.0 is on PyPI. New recipes: GEPA, SDFT, SDPO and Meta-Harness. Reef serves the OpenAI Responses API at /v1/responses #528.
- 2026-08: Reef is open source. Read the launch post.
🎯 When to use Reef
Use Reef when you want your agent to keep improving simply by learning from how you interact with your agent.
| Your goal | Learning path | What you need |
|---|
| Keep getting stronger model designed for you | Model weight training | A trainable model, a supported GPU stack, and feedback your recipe can use |
| Get your harness to self-improve | Harness optimization | A model endpoint, representative tasks, and an evaluator; no local training GPUs |
| Scientific discoveries | Test-time training | An execution environment, a correctness checker, and a measurable objective |
🧩 How Reef fits your stack
| Ability | Inference engine (vLLM, SGLang, …) | RL training framework (Slime, veRL, AReaL, …) | Reef |
|---|
| Serves live traffic | ✅ | ❌ | ✅ |
| Trains weights | ❌ | ✅ | ✅ |
| Version management | ❌ | ❌ | ✅ |
| Stays live through updates | ❌ | ❌ | ✅ |
| Evolves beyond weights (skills, harness) | ❌ | ❌ | ✅ |
🔄 How it works
Reef processes each learning cycle in four steps. The table also shows which
modules implement each step.
| Step | What happens | Where it lives |
|---|
| 1 · Serve | Serve agent requests and record interactions. | service/ — agent requests and interaction recordsinference/ — inference and artifact updates |
| 2 · Observe | Match feedback to recorded interactions. | storage/records.py — stored interactions and feedbackrecord2sample/ — feedback matching, eligibility, and samples |
| 3 · Grow | Produce an update from eligible records. | recipe/ — recipe integrationtrain/ — the training pipeline, job data, and training jobs |
| 4 · Commit | Apply the configured selection policy and publish accepted updates. | evaluation/ — candidate evaluationartifact/ — version history and snapshot components |
📦 Installation
💡 Note
Reef's artifact and checkpoint functionality requires the git-lfs system
package. Reef initializes Git LFS locally for its artifact repositories.
We recommend uv for managing packages, and the
commands below use it.
From PyPI
uv venv && source .venv/bin/activate
uv pip install reef-infra
python3 -c "import reef; print(reef.__version__)"
From source
git lfs install
git clone https://github.com/Human-Agent-Society/reef.git
cd reef
uv venv && source .venv/bin/activate
uv pip install -e .
python3 -c "import reef; print(reef.__version__)"
Use the source checkout for development and for the training examples below.
🔧 Using Reef
Reef supports two learning surfaces: model weights and agent harnesses.
The deployment's recipe determines which surface its scenarios update.
As a minimal example, start Reef as a pure inference server. This needs SGLang and
GPU-enabled PyTorch in the environment:
This is the beginning of the README. Read the rest at the original. Read the full README ↗
