RoboColosseum Client

Run robot evaluations for the RoboColosseum Open Track. The Client controls the robot, communicates with the Policy Server, and records and uploads evaluation results.

1. Install

Requirements: Python 3.10 or newer, uv, and FFmpeg for recording export.

git clone https://github.com/frodobots-org/colosseum-client.git
cd colosseum-client

2. Getting Started

cp configs/robot.yaml.example configs/robot.yaml

Edit configs/robot.yaml for your setup. See the Configuration example for configuration details.

Install first, then run:

uv sync --extra <robot_type>
.venv/bin/colosseum-robot configs/robot.yaml

Or install and run in one command:

uv run --extra <robot_type> colosseum-robot configs/robot.yaml

Follow the prompts to enter an instruction or select a task, start the trial, and submit the evaluation. Recordings are saved under eval_runs/ by default. After a successful LeRobot export, the trial's source PNGs are deleted, even when upload is skipped. Videos, Parquet, metadata and frames.jsonl remain for upload retries. Failed exports retain PNGs. The final observation without an action retains its numeric record only; its image is not included in the dataset.

Supported robots

Robot	robot_type	Status
Franka	franka	Implemented
SO-ARM101	so101	LeRobot driver and setup
Bimanual YAM	yam	I2RT driver and setup; hardware validation pending
Unitree G1	g1	TODO
Earth Rover Mini+	earth_rover_mini_plus	FrodoBots platform link and setup; physical validation pending

Adding a new robot

YAM local evaluations with VLAs and LLMs: record, review and submit

YAM Eval Station (GUI)

Once the one-time YAM setup is done, including gripper calibration with scripts/calibrate_yam_gripper.py, one browser page replaces the separate terminals for the Policy Server, recording and review:

uv pip install --python .venv/bin/python -e '.[gui]'   # once
.venv/bin/python scripts/eval_gui.py
# open http://127.0.0.1:8770

• Setup checks the GPU, CAN, both arms, the 3 cameras, the Router, the API keys and the Policy Server, and shows a green tick for each one that passes. It starts the Policy Server for you, and Ctrl-C stops it and frees the GPU.
• Record takes the task, then runs A and B with live camera frames, a step counter and a STOP button (or Esc).
• Review collects scores, a preference and your observations, which Claude Haiku turns into written feedback, then submits to the Router or discards the evaluation.

It defaults to joint control (configs/robot.yam-joint-llm.yaml). For EE control, pass --config and --server-config; see EE control. A software STOP is not an emergency stop, so keep the physical stop within reach.

Reference

Evaluation flow & API reference

LLM relay credentials

The Router supplies each LLM's API URL; no Client base-URL setting is needed. llm_api_keys.openai, .xai, and .anthropic select credentials by the assigned model name (gpt-*, grok-*, claude-*), including when all three use one relay host. Use that relay's respective keys, not unrelated official-provider keys. For example:

llm_api_keys:
  openai: env:YHLXJ_OPENAI_KEY
  xai: env:YHLXJ_XAI_KEY
  anthropic: env:YHLXJ_ANTHROPIC_KEY
deadline_ms: 120000

env: resolves process environment variables; it does not load .env by itself. Literal key values in a private YAML config are also supported. The Client sends only capability names/legacy URLs to Router, never keys. Upgrade Router before this Client: /next now includes llm_providers.