NVIDIA Isaac Capture
Shared capture infrastructure for teleoperation and egocentric data.
Overview
Isaac Capture provides shared infrastructure for collecting human and robot demonstration data.
It brings together two components:
- Isaac Teleop supports robot-in-the-loop capture through teleoperation, including Teleop Session and Remote Assist.
- Ego Pipeline supports human and egocentric data capture through Ego Capture and Ego Post Processing.
Both components use a shared core of device plugins, retargeters, and data utilities for recording,
replaying, and converting data. Isaac Teleop remains a component of Isaac Capture, with its existing
functionality preserved.
Key Features
- Unified stack for sim & real teleoperation
- Standardized device interface
- Flexible retargeting framework
Isaac Teleop
- Currently supported use cases
- Use XR headsets for gripper / tri-finger hand manipulation
- Use XR headsets with gloves for dex-hand manipulation
- Seated full body loco-manipulation (Homie)
- Tracking based full body loco-manipulation with GR00T-WholeBodyControl (SONIC)
- Upcoming use cases
- Teleoperate using only non-XR devices (e.g. gamepad, Gello, haply, etc.)
- Teleoperate cloud based robotics simulations
- Remote teleoperation with camera streaming to the desktop
- Remote teleoperation with immersive camera streaming to XR headsets
Ego Pipeline
- Capture egocentric data without a robot in the loop.
- Reconstruct hand and camera poses from egocentric video with the
Ego Post Processing pipeline.
Quick Start
Documentation
Our documentation page provides everything you need to get started, including detailed tutorials and step-by-step guides. Follow these links to learn more:
Check a Device Integration
The CTS records one scripted take from a hand-tracking or full-body device and checks it against the data a retargeter needs; start with the hand or full-body guide.
Install & Run Isaac Lab
The latest NVIDIA Isaac Lab release, 3.0.0 EA, includes Isaac Capture, previously released as Isaac Teleop.
To get started, please refer to Isaac Lab's Installation guide for more details. Then follow the CloudXR teleoperation in Isaac Lab to get started with Teleoperation in Sim.
Licensing
The default license is Apache-2.0.
EgoPost is licensed under OpenMDW-1.1.
Third-party materials may carry their own licenses.
Separately retrieved optional materials
The optional isaaccapture[ovxr] extra retrieves OVXR and its OVRTX/OVStage
dependencies. The separate isaaccapture[desktop-mirror] extra retrieves OVUI.
These packages are downloaded separately, rather than bundled in the Isaac Capture
wheel. Their licenses apply independently of Isaac Capture's Apache-2.0 license.
OVXR, OVRTX and OVStage are NVIDIA proprietary software. OVUI is Apache-2.0.
See package sources and license details.
Review the applicable package licenses and terms before installing or using these
optional dependencies.
Existing CloudXR native libraries are bundled in Isaac Capture wheels and remain subject
to their separate NVIDIA license and EULA acceptance.
