Introduction

VSLAM-LAB is designed to simplify the development, evaluation, and application of Visual SLAM (VSLAM) systems. This framework enables users to compile and configure VSLAM systems, download and process datasets, and design, run, and evaluate experiments — all from a single command line!

Why Use VSLAM-LAB?

• Unified Framework: Streamlines the management of VSLAM systems and datasets.
• Ease of Use: Run experiments with minimal configuration and single command executions.
• Broad Compatibility: Supports a wide range of VSLAM systems and datasets.
• Reproducible Results: Standardized methods for evaluating and analyzing results.

Getting Started

To ensure all dependencies are installed in a reproducible manner, we use the package management tool pixi. If you haven't installed pixi yet, please run the following command in your terminal:

curl -fsSL https://pixi.sh/install.sh | bash

After installation, restart your terminal or source your shell for the changes to take effect. For more details, refer to the pixi documentation.

If you already have pixi remember to update: pixi self-update

Clone the repository and navigate to the project directory:

git clone https://github.com/VSLAM-LAB/VSLAM-LAB.git && cd VSLAM-LAB

Quick Demo

You can now execute any baseline on any sequence from any dataset within VSLAM-LAB using the following command:

pixi run demo <baseline> <dataset> <sequence> <mode>

For a full list of available systems and datasets, see the VSLAM-LAB Supported Baselines and Datasets. Example commands:

pixi run demo mast3rslam eth table_3 mono
pixi run demo droidslam rgbdtum rgbd_dataset_freiburg1_xyz rgbd
pixi run demo orbslam2 kitti 04 stereo
pixi run demo pycuvslam euroc MH_01_easy stereo-vi

To change the paths where VSLAM-LAB-Benchmark and/or VSLAM-LAB-Evaluation data are stored (for example, to /media/${USER}/data), use the following commands:

pixi run set-benchmark-path /media/${USER}/data
pixi run set-evaluation-path /media/${USER}/data

Configure your own experiments

With VSLAM-LAB, you can easily design and configure experiments using a YAML file and run them with a single command. To run the experiment demo, execute the following command:

pixi run vslamlab configs/exp_vslamlab.yaml              # resumes: runs already done are kept
pixi run vslamlab configs/exp_vslamlab.yaml --overwrite  # deletes the previous results first
VSLAMLAB_HEADLESS=1 pixi run vslamlab configs/exp_vslamlab.yaml  # unattended: figures are saved, no plot windows open

Experiments in VSLAM-LAB are sequences of entries in a YAML file (see example configs/exp_vslamlab.yaml):

exp_demo_droidslam:
  Config: config_vslamlab.yaml               # YAML file containing the sequences to be run
  NumRuns: 1                                 # Maximum number of executions per sequence
  Parameters: {verbose: 1, rgb_idx: [0,250]} # Vector with parameters that will be input to the baseline executable
  Module: droidslam                          # droidslam/monogs/orbslam2/mast3rslam/dpvo/...

exp_demo_orbslam2:
  Config: config_vslamlab.yaml
  NumRuns: 1
  Parameters: {verbose: 1, rgb_idx: [0,250]}
  Module: orbslam2

exp_demo_colmap:
  Config: config_vslamlab.yaml
  NumRuns: 1
  Parameters: {verbose: 1, rgb_idx: [0,250], matcher_type: exhaustive, matching_type: aliked_lightglue}  # baseline-specific keys: see Baselines/colmap/README.md
  Module: colmap

Frame selection: Parameters also accepts four optional flags to control which RGB frames of a sequence are fed to the baseline. They are independent and stack in this order — rgb_idx slices the frame range first, rgb_step then keeps 1 frame every n of what remains, rgb_max truncates that result to at most that many frames, and rgb_vpr finally downsamples to at most that many frames by visual dissimilarity (VPR) rather than a fixed stride, so visually redundant frames are dropped preferentially. If none are set, every frame of the sequence is used.

Parameter	Effect
rgb_idx: [start, end]	Keep only frames start..end (inclusive)
rgb_step: n	Keep 1 frame every n frames of what remains
rgb_max: n	Truncate the result to at most n frames
rgb_vpr: n	Downsample the result to at most n frames, chosen by VPR dissimilarity - computes the sequence's VPR distance matrix via pixi run vpr <dataset> <sequence> first if it isn't already cached

exp_vslamlab:
  Config: config_vslamlab.yaml
  NumRuns: 1
  Parameters: {verbose: 1, rgb_idx: [0, 250], rgb_step: 3, rgb_max: 50, rgb_vpr: 30}
  Module: droidslam

Config files are YAML files containing the list of sequences to be executed in the experiment (see example configs/config_vslamlab.yaml):

eth:
- table_3

rgbdtum:
- 'rgbd_dataset_freiburg1_xyz'

For a full list of available VSLAM systems and datasets, refer to the section VSLAM-LAB Supported Baselines and Datasets.

VSLAM-LAB Pipeline Commands

In addition to running the full automated pipeline, VSLAM-LAB provides modular commands to interact directly with datasets and baselines. For a comprehensive list of all available commands consult Wiki: Command-line Interface

pixi run install-baseline <baseline>                     # Example: pixi run install-baseline droidslam
pixi run download-sequence <dataset> <sequence>          # Example: pixi run download-sequence eth table_3
pixi run run-exp <exp_yaml>                              # Example: pixi run run-exp configs/exp_vslamlab.yaml
pixi run evaluate <exp_yaml>                             # Example: pixi run evaluate configs/exp_vslamlab.yaml
pixi run compare <exp_yaml>                              # Example: pixi run compare configs/exp_vslamlab.yaml
pixi run rerun-vis <dataset> <sequence>                  # Example: pixi run rerun-vis eth table_3 (Rerun viewer of a sequence)

Run a VSLAM system online from an iPhone

The ARStream app streams RGB, intrinsics and ARKit's pose from an iPhone over USB or Wi-Fi. record-iphone turns a recording into a sequence of the iphone dataset (see the Tools table); stream-iphone relays the live stream over ZeroMQ so any number of subscribers (a baseline's execute-mono-stream, the viewer, your own Python/C++ code) consume it at once:

pixi run stream-iphone --run orbslam3-dev                # relay + ORB-SLAM3-DEV (or orbslam2-dev) online in one go: Ctrl-C stops the stream, the baseline saves ../VSLAM-LAB-Evaluation/stream_orbslam3-dev/00000_KeyFrameTrajectory.csv
pixi run stream-iphone --run orbslam3-dev --view         # ... plus the Rerun viewer as another subscriber (--duration 30, --verbose 0, --exp-folder <dir> are also available)
pixi run stream-iphone --run orbslam2 --mode rgbd        # LiDAR phone: RGB-D online with ARKit's depth (orbslam2/orbslam3 and their -dev variants), results in ../VSLAM-LAB-Evaluation/stream_orbslam2_rgbd
pixi run stream-iphone                                   # relay only: phone over USB (or --wifi / --host <ip>) -> tcp://*:5556, writes ../VSLAM-LAB-Benchmark/IPHONE/stream/calibration.yaml from the first frame
pixi run view-iphone --sub                               # the Rerun viewer as a subscriber
pixi run execute-mono-stream stream_url:tcp://localhost:5556 calibration_yaml:<abs path> exp_folder:<abs dir> settings_yaml:vslamlab_orbslam3-dev_settings.yaml   # a baseline's stream entry point by hand (runs inside Baselines/ORB-SLAM3-DEV)

Subscribers keep only the newest frame by default, so a system slower than the stream skips frames instead of lagging; the relay sends an end-of-stream message when it stops and the baseline then saves its KeyFrameTrajectory.csv as usual. A baseline joins --run by providing an execute-<mode>-stream entry point taking stream_url: instead of sequence_path:/rgb_csv: (see BaselineVSLAMLAB.build_stream_command). Writing a subscriber: arstream.Subscriber (Python) or arstream/arstream.hpp (C++, conda package libarstream), see the ARStream README.

Add a new VSLAM Dataset

Expand the evaluation suite by integrating custom datasets. If you're using Claude Code, run the add-dataset skill (/add-dataset <name> <source>) to walk through the full workflow end to end. Otherwise, follow the instructions in Wiki: Integrate a new VSLAM Dataset.

Add a new VSLAM Baseline

Incorporate new algorithms into the framework. Follow the guide in Wiki: Integrate a new VSLAM Baseline. Benchmark your method against state-of-the-art baselines across all supported datasets.

For a reference implementation, see the VGGT-SLAM integration in commit 259f7ae.

License

VSLAM-LAB is released under a LICENSE.txt. For a list of code dependencies which are not property of the authors of VSLAM-LAB, please check docs/Dependencies.md.

Citation

If you're using VSLAM-LAB in your research, please cite:

@INPROCEEDINGS{fontan2025vslam,
  author={Fontan, Alejandro and Fischer, Tobias and Civera, Javier and Milford, Michael},
  booktitle={2025 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)},
  title={VSLAM-LAB: A Comprehensive Framework for Visual SLAM Methods and Datasets},
  year={2025},
  volume={},
  number={},
  pages={9013-9020},
  keywords={Visualization;Simultaneous localization and mapping;Robot vision systems;Standardization;Benchmark testing;Programming;Reproducibility of results;Intelligent robots;Faces;Software development management},
  doi={10.1109/IROS60139.2025.11247218}
}

VSLAM-LAB Supported Baselines and Datasets

For full per-dataset details — license, access requirements, and publication/BibTeX citation — see Datasets/extra-files/dataset_table.xlsx.

Baselines	System	Modes	License	Label	Conda Pkg	Camera Models
AllFeature-VSLAM	VSLAM	mono rgbd	GPLv3	allfeature	✅	pinhole radtan4 radtan5
VGGT-SLAM 2.0	VSLAM	mono	BSD-2	vggtslam	✅	pinhole
MASt3R-SLAM	VSLAM	mono	CC BY-NC-SA 4.0	mast3rslam	✅	pinhole radtan4 radtan5
DPVO	VSLAM	mono	License

[!NOTE] -dev variants: orbslam2-dev, orbslam3-dev, droidslam-dev, dpvo-dev, mast3rslam-dev, vggtslam-dev, allfeature-dev, okvis2-dev and monogs-dev clone the baseline's source into Baselines/<Name>-DEV/ and build it locally instead of installing the conda package, with the same modes and camera models - use them to modify a baseline. monogs-dev exists only in this form.

Tools	Features	Label	Modes	Camera Models
Stray Scanner App	📸🤳	strayscanner	mono rgbd	pinhole
Pupil Labs Neon	📸🥽	pupil-labs	mono(-vi)	pinhole
Local video files	📸	videos	mono	pinhole unknown
YouTube videos add one with the add-youtube-video skill	📸	youtube	mono	pinhole radtan4 unknown
iPhone (ARStream app) record one with pixi run record-iphone [<sequence>] (USB or --wifi); ARKit's VIO pose as groundtruth	📸🤳	iphone	mono	pinhole
; ARKit's sceneDepth (256×192, confidence-masked, upsampled to the RGB size) as

Real / Synthetic : 📸 / 💻

Indoor / Outdoor / Underwater / Intracorporeal : 🏠 / 🏞️ / 🌊 / 🫀

Handheld / Headmounted / Vehicle / UAV / Robot : 🤳 / 🥽 / 🚗 / 🚁 / 🤖

Section dividers only (not per-row features) — Construction Site / Planetary-analog Terrain : 🏗️ / 🪐