MetaBox: A Benchmark Platform for Meta-Black-Box Optimization with Reinforcement Learning

MetaBox is the first benchmark platform expressly tailored for developing and evaluating MetaBBO-RL methods. MetaBox offers a flexible algorithmic template that allows users to effortlessly implement their unique designs within the platform. Moreover, it provides a broad spectrum of over 300 problem instances, collected from synthetic to realistic scenarios, and an extensive library of 19 baseline methods, including both traditional black-box optimizers and recent MetaBBO-RL methods. Besides, MetaBox introduces three standardized performance metrics, enabling a more thorough assessment of the methods.

Installations

You can access all MetaBox files with the command:

git clone [email protected]:GMC-DRL/MetaBox.git
cd MetaBox

Citing MetaBox

The PDF version of the paper is available here. If you find our MetaBox useful, please cite it in your publications or projects.

@inproceedings{metabox,
author={Ma, Zeyuan and Guo, Hongshu and Chen, Jiacheng and Li, Zhenrui and Peng, Guojun and Gong, Yue-Jiao and Ma, Yining and Cao, Zhiguang},
title={MetaBox: A Benchmark Platform for Meta-Black-Box Optimization with Reinforcement Learning},
booktitle = {Advances in Neural Information Processing Systems},
year={2023},
volume = {36}
}

Requirements

Python >=3.7.1 with the following packages installed:

• numpy==1.21.2
• torch==1.9.0
• matplotlib==3.4.3
• pandas==1.3.3
• scipy==1.7.1
• scikit_optimize==0.9.0
• deap==1.3.3
• tqdm==4.62.3
• openpyxl==3.1.2

Quick Start

• To obtain the figures in our paper, run the following commands:

  cd for_review
  python paper_experiment.py

  then corresponding figures will be output to for_revivew/pics.

  ---

  The quick usage of the four main running interfaces is listed as follows, in the following command, we specifically take RLEPSO as an example.

  Firstly, get into the main code folder, src:

  cd ../src

• To trigger the entire workflow, including train, rollout and test, run the following command:

  python main.py --run_experiment --problem bbob --difficulty easy --train_agent RLEPSO_Agent --train_optimizer RLEPSO_Optimizer

• To trigger the standalone process of training:

  python main.py --train --problem bbob --difficulty easy --train_agent RLEPSO_Agent --train_optimizer RLEPSO_Optimizer 

• To trigger the standalone process of testing:

  python main.py --test --problem bbob --difficulty easy --agent_load_dir agent_model/test/bbob_easy/ --agent_for_cp RLEPSO_Agent --l_optimizer_for_cp RLEPSO_Optimizer --t_optimizer_for_cp DEAP_CMAES Random_search

Documentation

For more details about the usage of MetaBox, please refer to MetaBox User's Guide.

Datasets

At present, three benchmark suites are integrated in MetaBox:

• Synthetic contains 24 noiseless functions, borrowed from coco:bbob with original paper.
• Noisy-Synthetic contains 30 noisy functions, borrowed from coco:bbob-noisy with original paper.
• Protein-Docking contains 280 problem instances, which simulate the application of protein docking as a 12-dimensional optimization problem, borrowed from LOIS with original paper.

Baseline Library

7 MetaBBO-RL optimizers, 1 MetaBBO-SL optimizer and 11 classic optimizers have been integrated into MetaBox. They are listed below.

Supported MetaBBO-RL optimizers:

Name	Year	Related paper
DE-DDQN	2019	Deep reinforcement learning based parameter control in differential evolution
QLPSO	2019	A reinforcement learning-based communication topology in particle swarm optimization
DEDQN	2021	Differential evolution with mixed mutation strategy based on deep reinforcement learning
LDE	2021	Learning Adaptive Differential Evolution Algorithm From Optimization Experiences by Policy Gradient
RL-PSO	2021	Employing reinforcement learning to enhance particle swarm optimization methods
RLEPSO	2022	RLEPSO:Reinforcement learning based Ensemble particle swarm optimizer✱
RL-HPSDE	2022	Differential evolution with hybrid parameters and mutation strategies based on reinforcement learning
GLEET	2024	Auto-configuring Exploration-Exploitation Tradeoff in Evolutionary Computation via Deep Reinforcement Learning
SYMBOL	2024	Symbol: Generating Flexible Black-Box Optimizers through Symbolic Equation Learning
RL-DAS	2024	Deep Reinforcement Learning for Dynamic Algorithm Selection: A Proof-of-Principle Study on Differential Evolution

Supported MetaBBO-SL optimizer:

Name	Year	Related paper
RNN-OI	2017	Learning to learn without gradient descent by gradient descent

Supported MetaBBO-NE optimizer:

Name	Year	Related paper
LES	2023	Discovering evolution strategies via meta-black-box optimization

Supported classic optimizers:

Name	Year	Related paper
PSO	1995	Particle swarm optimization
DE	1997	Differential Evolution – A Simple and Efficient Heuristic for Global Optimization over Continuous Spaces
CMA-ES	2001	Completely Derandomized Self-Adaptation in Evolution Strategies
Bayesian Optimization	2014	Bayesian Optimization: Open source constrained global optimization tool for Python
GL-PSO	2015	Genetic Learning Particle Swarm Optimization
sDMS_PSO	2015	A Self-adaptive Dynamic Particle Swarm Optimizer
j21	2021	Self-adaptive Differential Evolution Algorithm with Population Size Reduction for Single Objective Bound-Constrained Optimization: Algorithm j21
MadDE	2021	Improving Differential Evolution through Bayesian Hyperparameter Optimization
SAHLPSO	2021	Self-Adaptive two roles hybrid learning strategies-based particle swarm optimization
NL_SHADE_LBC	2022	NL-SHADE-LBC algorithm with linear parameter adaptation bias change for CEC 2022 Numerical Optimization
Random Search	-	-

Note that Random Search is to randomly sample candidate solutions from the searching space.

Post-processing

In a bid to illustrate the utility of MetaBox for facilitating rigorous evaluation and in-depth analysis, as mentioned in our paper, we carry out a wide-ranging benchmarking study on existing MetaBBO-RL methods. The post-processed data is available in content.md.

Acknowledgements

The code and the framework are based on the repos DEAP, coco and Protein-protein docking V4.0.