NVIDIA Isaac Capture

Shared capture infrastructure for teleoperation and egocentric data.

---

Overview

Isaac Capture provides shared infrastructure for collecting human and robot demonstration data. It brings together two components:

• Isaac Teleop supports robot-in-the-loop capture through teleoperation, including Teleop Session and Remote Assist.
• Ego Pipeline supports human and egocentric data capture through Ego Capture and Ego Post Processing.

Both components use a shared core of device plugins, retargeters, and data utilities for recording, replaying, and converting data. Isaac Teleop remains a component of Isaac Capture, with its existing functionality preserved.

Key Features

• Unified stack for sim & real teleoperation
• Standardized device interface
• Flexible retargeting framework

Isaac Teleop

• Currently supported use cases
  • Use XR headsets for gripper / tri-finger hand manipulation
  • Use XR headsets with gloves for dex-hand manipulation
  • Seated full body loco-manipulation (Homie)
  • Tracking based full body loco-manipulation with GR00T-WholeBodyControl (SONIC)
• Upcoming use cases
  • Teleoperate using only non-XR devices (e.g. gamepad, Gello, haply, etc.)
  • Teleoperate cloud based robotics simulations
  • Remote teleoperation with camera streaming to the desktop
  • Remote teleoperation with immersive camera streaming to XR headsets

Ego Pipeline

• Capture egocentric data without a robot in the loop.
• Reconstruct hand and camera poses from egocentric video with the Ego Post Processing pipeline.

Quick Start

Documentation

Our documentation page provides everything you need to get started, including detailed tutorials and step-by-step guides. Follow these links to learn more:

• Architecture
• Quick installation steps
• How to build from source

Check a Device Integration

The CTS records one scripted take from a hand-tracking or full-body device and checks it against the data a retargeter needs; start with the hand or full-body guide.

Install & Run Isaac Lab

The latest NVIDIA Isaac Lab release, 3.0.0 EA, includes Isaac Capture, previously released as Isaac Teleop.

To get started, please refer to Isaac Lab's Installation guide for more details. Then follow the CloudXR teleoperation in Isaac Lab to get started with Teleoperation in Sim.

Licensing

The default license is Apache-2.0. EgoPost is licensed under OpenMDW-1.1. Third-party materials may carry their own licenses.