srbf runs symbolic regression methods on the same expressions, sampled the same way, and reads every
prediction with one judge. A method is plugged in with a small adapter, a YAML config describes the
evaluation, and four commands take it from a first smoke test to a report:
srbf new mymethod # scaffold an adapter for your method
srbf check -c config.yaml # a few real problems, step by step
srbf run -c config.yaml -v # the evaluation
srbf analyze -c config.yaml -o report
- 29 catalogs, 6,660 expressions: Feynman, FastSRB, Nguyen and the other classical suites, physics
collections and synthetic corpora, served by
symbolic-data.
- Any method, in its own environment: a worker is one Python file with a
fit() function, run
in the interpreter you name, with whatever torch, Julia or NumPy version your method needs.
- One judge: every prediction is parsed, evaluated on held-out points and compared with the ground truth in
one canonical form.
- Evaluations that scale: inline
!sweep ladders, whole-suite configs, resumable runs, and
sharding across GPUs with a checked merge.
- Statistics that fit the design: paired comparisons on the same expressions, rank analysis with
critical differences, bootstrap intervals.
- Stated provenance: every configuration carries a label for who chose it, and every result file
records what ran.
The published results are on the results explorer.
Install
pip install srbf # the framework, the metrics and the adapters of the worker protocol
pip install "srbf[baselines]" # plus what the PySR, NeSymReS and E2E adapters import
pip install "srbf[analysis]" # plus matplotlib, for the figures of `srbf analyze`
pip install "srbf[flash-ansr]" # plus flash-ansr, for the Flash-ANSR adapter
srbf needs Python 3.12 or newer and installs symbolic-data and simplipy with it.
Quickstart
No GPU and no model are needed for a first run: srbf new writes an adapter whose placeholder fits
a linear model.
export SRBF_ROOT=$PWD/bench # models, results and adapters live here
srbf new mymethod --python "$(which python)"
srbf check -c bench/adapters/mymethod/config.yaml # two real problems, every step named
srbf run -c bench/adapters/mymethod/config.yaml --experiment nguyen -v
srbf analyze -c bench/adapters/mymethod/config.yaml -o report # report/results.md and figures/
To evaluate a released Flash-ANSR model on one catalog at one budget:
git clone https://github.com/psaegert/srbf && cd srbf && pip install ".[flash-ansr]" && export SRBF_ROOT=$PWD
hf download psaegert/flash-ansr-v25.0-T8-3M --local-dir models/psaegert/flash-ansr-v25.0-T8-3M
srbf run -c configs/evaluation/scaling/flash-ansr-v25.0-T8-3M_srbf.yaml --experiment nguyen --sweep-filter ladder=32 -v
Documentation
srbf.readthedocs.io
| Guide | What it covers |
|---|
| Quickstart | both tours above, with their output |
| Concepts | ground truth, problem, catalog, budget, judge: the vocabulary |
| Adding your method | the worker contract and the pull request |
| Running evaluations | the config, ladders, suites, resuming, sharding, clusters |
| Results | result files, deriving metrics, intervals, the report |
| Metrics | the definition of every metric |
| Paired comparisons | comparing two methods, ranking many |
| Benchmarks | the 29 catalogs and their sources, custom catalogs |
| Models | installing and configuring the built-in methods |
| Command line | every command and flag |
| Fairness | one protocol, upstream defaults, provenance labels, timing |
Citing
srbf accompanies Saegert & Köthe 2026, Breaking the Simplification Bottleneck in Amortized Neural
Symbolic Regression (ICML 2026), arXiv:2602.08885. If you
publish numbers on a catalog, cite its source as well
(Benchmarks).
Development
pip install -e ".[dev]"
pre-commit run --all-files
pytest tests
License
MIT (see LICENSE). Third-party attributions in THIRD_PARTY_LICENSES.
