OpenEnv: Agentic Execution Environments

An end-to-end framework for creating, deploying and using isolated execution environments for agentic RL, with a simple Gymnasium-style API.

What you get

• One API for every environment. reset(), step() and state(), sync or async, over a WebSocket.
• Isolated, deployable environments. Each environment is a Docker image that runs locally, on a cloud sandbox, or as a Hugging Face Space.
• 40+ ready-to-use environments, from games and coding sandboxes to browsers, finance and simulators, in the environment catalog.
• Train with your framework. TRL, Unsloth, SkyRL, ART, Oumi, torchforge, Miles and more (see Integrations).
• Train real coding agents. The Harbor integration runs Claude Code, Codex, OpenCode, mini-swe-agent and other harnesses on Harbor tasks, and captures the exact tokens for RL.
• Serve tools to agents. MCP environments expose their tools to agents over /mcp in production mode, while training keeps using step() and rewards.
• Rewards and evals built in. Compose rewards with rubrics and evaluate with Inspect AI.

Quick Start

Install the OpenEnv package:

pip install openenv

Install an environment client (e.g., Echo):

pip install git+https://huggingface.co/spaces/openenv/echo_env

Then use the environment:

import asyncio
from echo_env import CallToolAction, EchoEnv

async def main():
    # Connect to a running Space (async context manager)
    async with EchoEnv(base_url="https://openenv-echo-env.hf.space") as client:
        # Reset the environment
        result = await client.reset()
        print(result.observation.metadata["message"])  # "Echo environment ready!"

        # Send messages
        result = await client.step(
            CallToolAction(
                tool_name="echo_message",
                arguments={"message": "Hello, World!"},
            )
        )
        print(result.observation.result["data"])  # "Hello, World!"
        print(result.reward)

asyncio.run(main())

Synchronous usage is also supported via the .sync() wrapper:

from echo_env import CallToolAction, EchoEnv

# Use .sync() for synchronous context manager
with EchoEnv(base_url="https://openenv-echo-env.hf.space").sync() as client:
    result = client.reset()
    result = client.step(
        CallToolAction(
            tool_name="echo_message",
            arguments={"message": "Hello, World!"},
        )
    )
    print(result.observation.result["data"])

For a detailed quick start, check out the docs page.

Train an agent

Any training framework that can call an environment can train on it. Training with OpenEnv maps the ways to train and the frameworks that support each one.

• Environments as tools (white-box). The trainer runs the multi-turn tool loop and the environment supplies the tools and the reward. Each framework under Integrations has its own example. With TRL, GRPOTrainer takes an environment_factory: start with the Wordle GRPO tutorial or TRL's OpenEnv guide.
• Real agent harnesses (loop-owning). The agent runs its own loop, and OpenEnv's Harbor integration captures every model call as a framework-neutral TrainingTrace. TRL's AsyncGRPOTrainer trains on those captures today: see examples/async_grpo_harbor and The ultimate guide to multi-harness RL.

Build your own environment

openenv init my_env       # scaffold an environment
openenv validate my_env --level static --skip-build   # quick check against the OpenEnv contract
openenv push my_env       # deploy it to Hugging Face Spaces

For local development, run openenv serve my_env after installing its dependencies. This runs on the host and binds to 127.0.0.1; use Docker or Spaces for isolated training and deployment.

See Your First Environment and Deploying an Environment. openenv import wraps an existing environment from ORS/OpenReward or Verifiers, or an AgentEnv bundle.

Environments

A few to start with:

Environment	What it is
Echo	Minimal MCP environment, for learning the API and testing a deployment
Coding	Sandboxed Python execution with stdout, stderr and exit codes
TextArena (Wordle and more)	Text games for multi-turn RL
OpenSpiel	Board and card games from DeepMind's OpenSpiel
BrowserGym	Web navigation tasks (MiniWoB++, WebArena, ...)
Harbor	Harbor task datasets through coding-agent harnesses, with token capture for training

Browse all of them in the environment catalog, or on the OpenEnv Hub organization.

This is the beginning of the README. Read the rest at the original. Read the full README ↗