ROCK: Reinforcement Open Construction Kit
ROCK (Reinforcement Open Construction Kit) is a easy-to-use, and scalable sandbox environment management framework, primarily for agentic reinforcement learning environments. It provides tools for building, managing, and scheduling reinforcement learning environments, suitable for development, testing, and research scenarios.
ROCK adopts a client-server architecture, supports different levels of isolation mechanisms to ensure stable environment operation, and supports integration with various reinforcement learning training frameworks through SDK. ROCK not only supports traditional sandbox management functions but also is compatible with GEM-like protocols, providing standardized interfaces for reinforcement learning environments.
🚀 Get Started
Documents
Quick Start
Installation
Quick Start
Configuration
API References
Recommended:
Install from source (using uv), or install from PyPI.
To start the local admin server, make sure Docker and uv are installed and that you can pull the python:3.11 Docker image. If you’re using macOS, see the “Getting Started” guide—especially the “macOS startup” section.
# Clone repository
git clone https://github.com/alibaba/ROCK.git
cd ROCK
# Create virtual environment (using uv-managed Python)
uv venv --python 3.11 --python-preference only-managed
# Install dependencies using uv
uv sync --all-extras
# Activate virtual environment
source .venv/bin/activate
# for MacOS, change ROCK_WORKER_ENV_TYPE to uv
# export ROCK_WORKER_ENV_TYPE=uv
# Start admin server
rock admin start
PyPI Installation (Recommended for simple testing)
To install ROCK from PyPI (recommended only for simple testing):
# Install from PyPI
pip install rl-rock
# If using the pip runtime environment (for sandbox dependency installation),
# you may need to set runtime environment type to pip:
export ROCK_WORKER_ENV_TYPE=pip
# Start admin server
rock admin start
Notes: ROCK depends on Docker and uv tools for environment management.
-
Python Environment Configuration: To ensure ROCK can correctly mount the project and virtual environment along with its base Python interpreter, it is strongly recommended to use uv-managed Python environments to create virtual environments rather than system Python. This can be achieved through the --python-preference only-managed parameter.
-
Distributed Environment Consistency: In distributed multi-machine environments, please ensure that all machines use the same root Python interpreter for ROCK and uv Python configurations to avoid environment inconsistencies.
-
Dependency Management: Use the uv command to install all dependency groups, ensuring consistency between development, testing, and production environments.
-
Pip Source Installation: For pip source installation (e.g., pip install rl-rock), you need to set the ROCK_WORKER_ENV_TYPE=pip environment variable and ensure network access for the sandbox to install dependencies. See Configuration Documentation for more details on runtime environment options and environment variables.
-
OS Support: ROCK recommends managing environments on the same operating system, such as managing Linux image environments on a Linux system. However, it also supports cross-operating system level image management, for example, launching Ubuntu images on MacOS.
Using Env Protocol
ROCK is fully compatible with the GEM protocol, providing standardized environment interfaces:
import rock
import random
# Create environment using GEM standard interface
env_id = "game:Sokoban-v0-easy"
env = rock.make(env_id)
# Reset environment
observation, info = env.reset(seed=42)
while True:
# Interactive environment operations
action = f"\\boxed{{{random.choice(['up', 'left', 'right', 'down'])}}}"
observation, reward, terminated, truncated, info = env.step(action)
if terminated or truncated:
break
# Close environment
env.close()
Sandbox SDK Usage
import asyncio
from rock.actions import CreateBashSessionRequest
from rock.sdk.sandbox.client import Sandbox
from rock.sdk.sandbox.config import SandboxConfig
async def run_sandbox():
config = SandboxConfig(image="python:3.11", memory="8g", cpus=2.0)
sandbox = Sandbox(config)
await sandbox.start()
await sandbox.create_session(CreateBashSessionRequest(session="bash-1"))
result = await sandbox.arun(cmd="echo Hello Rock", session="bash-1")
await sandbox.stop()
if __name__ == "__main__":
asyncio.run(run_sandbox())
🚀 Core Features
- Multi-Protocol Action Support: Supports multiple action protocols including GEM, Bash, and Chat.
- Sandbox Runtime: Stateful runtime environments with multiple isolation mechanisms to ensure consistency and security
- Flexible Deployment: Supports different deployment methods for diverse environment requirements and Operating System
- Unified SDK Interface: Clean Python SDK for Env and Sandbox interaction
- Layered Service Architecture: Distributed Admin, Worker, and Rocklet architecture for scalable resource management
- Efficient Resource Management: Automatic sandbox lifecycle management with configurable resource allocation
📢 Updates
🛠️ System Architecture
ROCK Service Architecture
The service layer implements a distributed architecture with three core node roles:
- ROCK SDK: Environment development toolkit that provides development tools to assist developers in building, registering, deploying, and accessing Environments.
- ROCK CLI: Command-line interface tool that helps users manage and operate Environments and Services.
- ROCK Admin: The scheduling node responsible for deploying Environments as Sandboxes and managing Sandbox resource scheduling and allocation
- ROCK Worker: The working node that allocates machine physical resources to Sandboxes and executes the specific Sandbox runtime
- ROCK Rocklet: A lightweight proxy service component that handles SDK-to-Sandbox Action communication and supports external internet service access
- ROCK Envhub: Environment repository that provides registration and storage for Environment data.
Core Technologies
- Distributed Architecture: Multi-node design with Admin, Worker, and Rocklet components for scalability
- Runtime Isolation: Stateful sandbox runtimes with multiple isolation mechanisms
- Flexible Deployment: Support for different deployment methods for diverse environment requirements and Operating System
- Protocol Compatibility: Support for multiple interaction protocols
- Container Orchestration: Docker-based container management with resource allocation
GEM Protocol Support
ROCK maintains compatibility with GEM interfaces for reinforcement learning environments:
make(env_id): Create environment instancereset(seed): Reset environment statestep(action): Execute action and return results
GEM environments follow standard return formats:
# reset() returns
observation, info = env.reset()
# step() returns
observation, reward, terminated, truncated, info = env.step(action)
📄 Configuration
Server Configuration
# Activate virtual environment
source .venv/bin/activate
# Start Rock service, default local startup
rock admin start
Service Information: The ROCK Local Admin service runs by default on http://127.0.0.1:8080. You can access this address through your browser to view the management interface.
For additional configuration information such as logs and Rocklet service startup methods, please refer to Configuration
Development Environment Configuration
# Sync dependencies using uv
uv sync --all-extras --all-groups
# Run in virtual environment
source .venv/bin/activate
# Run tests
uv run pytest tests
🤝 Contribution
We welcome contributions from the community! Here's how to get involved:
Development Setup
- Fork the repository
- Create a feature branch
- Make your changes
- Add tests if applicable
- Submit a pull request
Reporting Issues
Please use the GitHub issue tracker to report bugs or suggest features.
Code Style
Follow existing code style and conventions. Please run tests before submitting pull requests.
📄 License
ROCK is distributed under the Apache License (Version 2.0). This product contains various third-party components under other open source licenses.
🙏 Acknowledgements
ROCK is developed by Alibaba Group. The rocklet component of our project is mainly based on SWE-ReX, with significant modifications and enhancements for our specific use cases. And we deeply appreciate the inspiration we have gained from the GEM project.
Special thanks to:
🤝 About [ROCK & ROLL Team]
ROCK is a project jointly developed by Taotian Future Living Lab and Alibaba AI Engine Team, with a strong emphasis on pioneering the future of Reinforcement Learning (RL). Our mission is to explore and shape innovative forms of future living powered by advanced RL technologies. If you are passionate about the future of RL and want to be part of its evolution, we warmly welcome you to join us!
For more information about ROLL, please visit:
Learn more about the ROCK & ROLL Team through our official channels below👇
