BBOB Landscape Atlas

Live: https://bjbodner.github.io/bbob-landscapes/

An interactive viewer of the 24 noiseless bbob benchmark functions of the COCO platform in two dimensions. Each landscape turns slowly, as a heat map or as an elevation map.

• Exact values. Every landscape is evaluated with cocoex on a 201 × 201 grid over the search domain [−5, 5]², for bbob instance 1 (the 2019 suite). Hover over the surface, or tap it, to read x, f(x) − f_opt and f(x) at a grid point.
• Two views, two scales. The heat map and the elevation map can each show f(x) − f_opt on a log or a linear scale. Contour lines are drawn at equal steps of the colour scale.
• The optimum. It is marked from COCO's own x_opt, with its f_opt in the footer.
• The classification. Each function shows its bbob group (separable; low or moderate conditioning; high conditioning and unimodal; multimodal with adequate global structure; multimodal with weak global structure) and pills for its modality, separability, local surface, conditioning and global structure, as described in the bbob function definitions [2].
• Navigation. Use the side index, the arrow keys, or a link to one function such as #f15. H and E switch the view, L switches the scale, and Space pauses the rotation. Drag to turn and tilt.

Rebuild

pip install -r requirements.txt
python build.py                 # writes index.html
python build.py --instance 7    # another bbob instance

build.py fills template.html with the grid values, stored as log10(f(x) − f_opt) quantised to 16 bits, and writes a single self-contained index.html. The page loads three.js r128 from cdnjs and its fonts from Google Fonts.

demo/record.mjs records the 30-second demo video (node demo/record.mjs video) and the link-preview image (node demo/record.mjs preview). It drives the page in headless Chrome on a fake clock, and needs ffmpeg.

Optimizer Race

race.html replays recorded runs of six optimizers on one bbob function in 2-D, evaluation by evaluation: AMM (preset c46 in its anytime mode), HCMA, SLSQP, Nelder-Mead, NGOpt and CMA-BIPOP. The animation starts at f15; its automatic sequence runs f15 to f24 and wraps back to f15. The first 14 functions finish within a few dozen evaluations, but they can still be picked from the index.

• How AMM runs here. AMM plans for the budget it is told.
  • In the race it runs c46 in its anytime mode (variant.anytime="replay"; AMM's research code at 5df4ef6b, docs/ANYTIME.md, not yet in a release). This mode plans 10 × D evaluations first and doubles the plan as the run goes on, so it needs no budget chosen per function.
  • Each AMM run shown is its best of 100 random seeds (race/replay_exp.py), ignoring any run that reaches 1e-8 in fewer than 4 evaluations. Every other optimizer ran once (HCMA: its single archived run).
  • With this selection AMM wins 106 of the 120 races against SLSQP and HCMA, and 86 against all five competitors. It reaches 1e-8 in all 120.
  • For comparison (race/tuning/replay/comparison.md):
    • The previous version, c46 with its budget tuned per function plus the best of 100 seeds, won 114 and 112.
    • The anytime mode with one run each wins 49 and 22.
  • Recording needs the research code on the path: PYTHONPATH=<research main>/src python race/record.py --only amm --seeds race/tuning/amm_seeds_replay.json (race/tuning/amm_race.json selects the mode).
  • Earlier settings, kept for reference:
    • The budget sweep (race/budget_sweep.py, race/sweep_report.py, race/tuning/sweep_matrix.md).
    • The joint budget and seed search (race/joint_opt.py).
    • One budget for every function (B = 600, chosen on instances 6-15) won 44 and 32 with one run each.
• On the landscape. Each × is one evaluation. The last 10 of each optimizer are shown, fading with age. The ring marks each optimizer's best point so far. It never fades, and it glides to a new point only when a better one is found.
• The track. Each optimizer's best f(x) − f_opt so far, on a log scale from 10 down to the finish line at 10⁻⁸ (COCO's final target).
• The finish. The first to cross wins. When every optimizer on the track has finished, or after 1000 × D = 2000 evaluations, the next function starts.
• Controls.
  • The seed (1-5) is the bbob instance.
  • Each optimizer can be switched on or off.
  • Heat map or elevation; muted or Atlas colours; speed.
  • Links like race.html#f15-s2 open one race.
• The data. race/data/f{f}_i{k}.json (about 120 kB each) holds the landscape and every recorded sample. It is written by , with the versions in .

The page fetches its data, so serve the folder rather than opening the file: python -m http.server, then open http://localhost:8000/race.html.

The renderer is shared with the Atlas:

• src/landscape.js: the three.js surface, 2-D fallback, overlay canvas and picking.
• src/bbob-meta.js: names and classification.
• src/race.js and src/race.css: the race. BBOBRace.mount(element, options) embeds it in another page.

demo/snap.mjs takes deterministic screenshots of either page on a fake clock.

References

1. N. Hansen, A. Auger, R. Ros, O. Mersmann, T. Tušar, D. Brockhoff, COCO: A platform for comparing continuous optimizers in a black-box setting (2021), Optimization Methods and Software 36(1).
2. N. Hansen, S. Finck, R. Ros, A. Auger, Real-Parameter Black-Box Optimization Benchmarking 2009: Noiseless Functions Definitions (2009), INRIA Research Report RR-6829.

License

Apache-2.0. three.js is MIT-licensed. COCO is BSD-licensed.