Robot Lobby

An multiplayer MuJoCo sandbox in the browser: spawn robots, drive them, pull them around, in one physics world shared by everyone connected (a Python server, e.g. on Binder), or alone with the physics in the page.

Live demo: https://yxzhan.github.io/robot-lobby/ (front end only: the physics runs in your browser, with part of the model library).

https://github.com/user-attachments/assets/242248f9-c4d9-415c-8fe0-4d691a56f08c

Physics from the official MuJoCo WASM bindings (@mujoco/mujoco), rendering with Filament.js (WebGL2, PBR, sun shadows, SSAO). No bundler — public/ is a static site. Its modules sit in public/js/ by what they are, each with its panel and its shipped defaults: app/ (the page, main.js), world/, net/ (network mode), render/, cameras/, controllers/ (policies, base/, ik/), drive/ (who drives each robot: keys, gamepad, goals; the Teleop panel), ros/, behavior/ and agent/; public/json/ holds only generated indexes.

UI Guide

• Camera: left-drag orbits, right-drag pans, the wheel zooms. Click an axis of the gizmo (bottom left) to look along it. The meters at the bottom right show the frame rate (click to cycle) and the sim speed.
• Spawning: drag a model from the Assets panel onto the floor. While dragging, the wheel or Q/E turns it, Shift+wheel or R/F raises it, Esc cancels.
• Selecting and pulling: double-click a body to select it (the floor deselects), Shift+drag to pull it.
• Driving: the Teleop panel drives the newest or double-clicked robot: WASD walk, QE turn, or drag the goal marker on the floor; a gamepad works too. Robot arms have IK markers to drag.
• Toolbar (top middle): Pause, Reset (every robot back to its spawn pose), Clear (remove everything), Enter VR / AR where the browser supports it.
• Panels, left: Assets (the model library), Stage (what is spawned: poses, joints; save, load, download and share stages), Physics; right: Teleop, Cameras (the robots' cameras), Render (quality vs. GPU load, floor, lighting, camera follow), Share (open the page on another device).
• Touch screens: one finger orbits, two pinch and pan; double tap selects, press and hold then drag pulls.

Local Development

In the browser alone (the physics in the page):

scripts/get_assets.sh   # the model library (mjcf_assets/, lite)
npm install      # postinstall copies the wasm/js runtimes into public/vendor/
npm start        # python3 scripts/serve.py 4007: public/ with Cache-Control: no-store

Open http://localhost:4007 (in JupyterHub: .../user/<name>/proxy/4007/ — all URLs are relative, so the path proxy works).

A world shared by every browser that opens it (the physics in Python):

pip install mujoco numpy aiohttp onnxruntime numba
npm run server   # python3 server/server.py 4007

Open http://localhost:4007. ?local in the URL keeps the page local even there.

The model library is mjcf_assets/ next to package.json, a repository of its own: scripts/get_assets.sh gets it (lite, the default: the models of the GitHub Pages site; main: all of them) and updates it. Run node scripts/vendor.mjs after adding models. Walking policies go in pretrained_policies/ (Policies).

Docs

• Spawning — the asset library, thumbnails, how the world is rebuilt on every spawn, memory
• Physics — the Physics panel: options, timestep, floor, lighting, sleep
• Teleop — driving, the goal marker, gamepads, scripted goals
• Agent — the Agent panel: Claude (your API key) or an Ollama model working the lobby with tools
• Policies — GR00T (G1), Playground (Spot, Go2), Microduck
• Mobile bases and IK chains
• ROS bridge — the ROS panel: joint states, TF, commands, cameras over rosbridge
• Real robots (robot_station/, a submodule) — a real Panda (later SO-101) follows the lobby robot over rosbridge: teleop from the Quest 3, the real robot as a ghost, demonstrations recorded
• Cameras — the robots' cameras
• Rendering — the Render panel, textures, materials
• VR and AR — WebXR controllers and menu
• Shared world — the Python server and network mode