RelBench

News

• Oct 2026 — Agentic leaderboard submissions: point your AI agent at the leaderboard README and it can submit end to end. Entries can now carry an optional repro link for reproducibility and can be edited.
• Aug 2026 — RelBench v3 released: datasets and tasks load straight from Hugging Face, a new leaderboard with automated submissions, and bug fixes (migration guide).
• Jun 2026 — Datasets, models, and community hub migrated to Hugging Face.
• Mar 2026 — RelBench v2 paper accepted at the ICLR 2026 DATA-FM workshop.
• Feb 2026 — Temporal Graph Benchmark integration: time-stamped event streams as relational schemas.
• Jan 2026 — RelBench v2 released: four new databases (SALT, RateBeer, arXiv, MIMIC-IV), 36 new tasks, and a new Autocomplete task type; plus 70+ CTU datasets via ReDeLEx and 7 from 4DBInfer.
• Jan 2026 — ReDeLEx integration: 70+ CTU relational databases via relbench[ctu].
• Sep 2024 — RelBench paper accepted at NeurIPS 2024 Datasets & Benchmarks Track.
• Jul 2024 — RelBench released: the first open benchmark for predictive ML on relational databases.
• May 2024 — Relational Deep Learning position paper accepted at ICML 2024.
• Nov 2023 — Relational Deep Learning introduced in a keynote by Jure Leskovec at the LoG Conference (slides).

Get Started

pip install relbench             # data + task loading
pip install "relbench[example]"  # + PyTorch Geometric & PyTorch Frame, for the GNN examples
pip install pyg-lib -f https://data.pyg.org/whl/torch-2.9.0+cpu.html  # neighbor sampling; use the index matching your torch/CUDA build

Load a dataset and a task — both come straight from the Hub, with no per-dataset code:

import relbench

dataset = relbench.load_dataset("rel-f1")   # or a HuggingFace 'org/repo[/subdir]', or a local path
db = dataset.get_db()                  # rows after test_timestamp are hidden

task = dataset.load_task("driver-position")   # dataset.get_task_names() lists them
train_table = task.get_table("train")  # train / val / test label tables
test_table  = task.get_table("test")   # the target column is hidden on test

# ... train any model on db + train_table, predict on the test entities ...
metrics = task.evaluate(test_pred)     # standard metric for the task type

dataset.val_timestamp / dataset.test_timestamp give the temporal split points. RelBench is framework-agnostic — bring any modeling stack. For a reference Graph Neural Network on PyTorch Geometric + PyTorch Frame, see relbench.modeling and the runnable scripts in examples/.

Tutorials

Open these directly in Google Colab — no setup required:

Tutorial	What it covers
Quickstart	Load a dataset/task, explore the schema, run a baseline
Training a GNN	A GNN baseline for an entity task (PyG + PyTorch Frame)

Leaderboard

The RelBench leaderboard ranks methods by their test-set performance, averaged over a fixed task set, with three independent boards: classification, regression, and recommendation. To submit, write one prediction CSV per task, validate and package them with python -m relbench.submit, and open a submission issue — or let python -m relbench.submit --submit do the upload and open the issue for you.

leaderboard/README.md has the rules and the full instructions, including the prediction CSV format for writing predictions with your own code.

BYOD (Bring Your Own Data)

You can easily express your own databases and tasks in the RelBench format. A dataset is a self-describing folder — a manifest.yaml (tables, keys, the foreign-key graph, the time splits), one plain parquet per table, and a tasks/ subdirectory — that you publish to the Hugging Face Hub. RelBench loads it straight from its org/repo[/subdir] address; there is no central registry to register with.

byod/README.md is the full walkthrough, and the published stanford-star/relbench-v1/rel-f1 is a complete worked example. For how RelBench's own databases were built, cleaned, and verified from their original sources, see provenance/.

If you upload your data in RelBench format to Hugging Face, please let us know by opening an issue / making a PR so we can list it here!

Citations

If you use RelBench, please cite the benchmark papers:

@inproceedings{relbench,
  title={RelBench: A Benchmark for Deep Learning on Relational Databases},
  author={Robinson, Joshua and Ranjan, Rishabh and Hu, Weihua and Huang, Kexin and Han, Jiaqi and Dobles, Alejandro and Fey, Matthias and Lenssen, Jan Eric and Yuan, Yiwen and Zhang, Zecheng and He, Xinwei and Leskovec, Jure},
  booktitle={Advances in Neural Information Processing Systems},
  year={2024}
}

@inproceedings{relbenchv2,
  title={RelBench v2: A Large-Scale Benchmark and Repository for Relational Data},
  author={Gu, Justin and Ranjan, Rishabh and Kanatsoulis, Charilaos and Tang, Haiming and Jurkovic, Martin and Hudovernik, Valter and Znidar, Mark and Chaturvedi, Pranshu and Shroff, Parth and Li, Fengyu and Leskovec, Jure},
  booktitle={3rd Workshop on Navigating and Addressing Data Problems for Foundation Models (DATA-FM) at ICLR 2026},
  year={2026},
  note={arXiv:2602.12606 [cs.LG]},
  url={https://arxiv.org/abs/2602.12606}
}

Datasets sourced from external repositories (CTU/ReDeLEx, 4DBInfer, TGB) carry their own citations on their Hugging Face dataset cards.