synthetic-diff.md

The overall comparison on optimization performance among all baselines in our Repo. For each problem in Synthetic difficult suites, we test each algorithms for $51$ independent runs and average the test results for presentation. ''Obj'' (smaller is better) indicates the final global best value. ''Gap'' (smaller is better) is the optimization gap away from the SOTA optimizer, which is CMA-ES algorithm in this paper. ''FEs'' indicates how many function evaluations an optimizer takes to get the ''Obj''. Under the consideration of runtime, we assign BayesianOptimizer 100 FEs. For RNN-OI, we also set the maxFEs to 100 respecting the original paper. Therefore, their ''FEs'' and ''Gap'' are not comparable.

Problem

Rastrigin_F15

Schaffers

Ellipsoidal_high_cond

Different_Powers

Bent_Cigar

Gallagher_21Peaks

Rosenbrock_rotated

Discus

Katsuura

Buche_Rastrigin

Rosenbrock_original

Rastrigin

Schaffers_high_cond

Weierstrass

Lunacek_bi_Rastrigin

Linear_Slope

Schwefel

Sphere

metric

Obj

Gap

FEs

Obj

Gap

FEs

Obj

Gap

FEs

Obj

Gap

FEs

Obj

Gap

FEs

Obj

Gap

FEs

Obj

Gap

FEs

Obj

Gap

FEs

Obj

Gap

FEs

Obj

Gap

FEs

Obj

Gap

FEs

Obj

Gap

FEs

Obj

Gap

FEs

Obj

Gap

FEs

Obj

Gap

FEs

Obj

Gap

FEs

Obj

Gap

FEs

Obj

Gap

FEs

DE_DDQN

3.341e+1 (5.982e+0)

0.271

2.000e+4 (0.000e+0)

1.334e-3 (5.618e-3)

0.000

2.000e+4 (0.000e+0)

4.462e+1 (2.191e+2)

0.001

1.642e+4 (1.612e+3)

1.128e-3 (6.236e-3)

0.000

1.427e+4 (2.616e+3)

1.081e+4 (5.039e+4)

0.001

2.000e+4 (0.000e+0)

6.768e+0 (7.738e+0)

-0.349

1.899e+4 (3.471e+3)

1.583e+0 (3.318e+0)

0.001

1.950e+4 (7.097e+2)

3.871e-1 (1.359e+0)

0.008

1.845e+4 (2.781e+3)

1.337e+0 (2.574e-1)

0.844

2.000e+4 (0.000e+0)

3.624e+1 (5.494e+0)

0.181

2.000e+4 (0.000e+0)

8.719e-1 (2.291e+0)

0.000

1.959e+4 (5.403e+2)

3.180e+1 (5.477e+0)

0.241

2.000e+4 (0.000e+0)

2.924e-3 (6.208e-3)

0.000

2.000e+4 (0.000e+0)

9.225e+0 (1.856e+0)

0.926

2.000e+4 (0.000e+0)

4.326e+1 (5.831e+0)

0.329

2.000e+4 (0.000e+0)

1.731e-1 (1.224e+0)

0.003

9.533e+3 (2.484e+3)

1.877e+0 (3.811e-1)

0.001

2.000e+4 (0.000e+0)

8.272e-9 (1.423e-9)

0.000

1.095e+4 (5.183e+2)

DEDQN

2.065e+2 (3.400e+1)

1.813

2.000e+4 (0.000e+0)

8.541e+0 (1.316e+0)

1.991

2.000e+4 (0.000e+0)

4.306e+5 (1.932e+5)

8.711

2.000e+4 (0.000e+0)

1.300e+1 (2.769e+0)

2.763

2.000e+4 (0.000e+0)

3.086e+7 (1.173e+7)

3.135

2.000e+4 (0.000e+0)

6.287e+1 (9.941e+0)

6.008

2.000e+4 (0.000e+0)

1.264e+4 (5.231e+3)

5.702

2.000e+4 (0.000e+0)

7.702e+2 (8.658e+2)

15.150

2.000e+4 (0.000e+0)

3.458e+0 (8.248e-1)

3.164

2.000e+4 (0.000e+0)

4.356e+2 (1.153e+2)

2.808

2.000e+4 (0.000e+0)

1.423e+4 (6.853e+3)

6.382

2.000e+4 (0.000e+0)

2.236e+2 (4.656e+1)

1.967

2.000e+4 (0.000e+0)

2.966e+1 (6.526e+0)

2.034

2.000e+4 (0.000e+0)

2.160e+1 (5.361e+0)

2.704

2.000e+4 (0.000e+0)

1.686e+2 (1.811e+1)

1.689

2.000e+4 (0.000e+0)

8.269e+1 (1.679e+1)

1.591

2.000e+4 (0.000e+0)

6.412e+3 (3.474e+3)

18.372

2.000e+4 (0.000e+0)

3.401e+1 (9.416e+0)

2.965

2.000e+4 (0.000e+0)

RL_HPSDE

5.824e+1 (9.864e+0)

0.492

2.025e+4 (0.000e+0)

1.285e+0 (3.542e-1)

0.300

2.025e+4 (0.000e+0)

2.155e+3 (1.644e+3)

0.044

2.025e+4 (0.000e+0)

1.538e-1 (1.023e-1)

0.033

2.025e+4 (0.000e+0)

1.158e+5 (8.269e+4)

0.012

2.025e+4 (0.000e+0)

5.183e+0 (4.787e+0)

-0.529

2.025e+4 (0.000e+0)

3.856e+1 (2.007e+1)

0.017

2.025e+4 (0.000e+0)

1.220e+1 (1.232e+1)

0.240

2.025e+4 (0.000e+0)

1.777e+0 (3.109e-1)

1.326

2.025e+4 (0.000e+0)

7.332e+1 (9.783e+0)

0.425

2.025e+4 (0.000e+0)

4.681e+1 (2.682e+1)

0.021

2.025e+4 (0.000e+0)

5.668e+1 (9.394e+0)

0.465

2.025e+4 (0.000e+0)

4.809e+0 (1.627e+0)

0.330

2.025e+4 (0.000e+0)

1.097e+1 (2.749e+0)

1.177

2.025e+4 (0.000e+0)

6.877e+1 (8.625e+0)

0.605

2.025e+4 (0.000e+0)

0.000e+0 (0.000e+0)

0.000

6.114e+3 (1.397e+3)

2.421e+0 (2.366e-1)

0.003

2.025e+4 (0.000e+0)

1.010e-1 (6.139e-2)

0.009

2.025e+4 (0.000e+0)

LDE

1.570e+1 (4.845e+0)

0.113

2.000e+4 (0.000e+0)

9.637e-3 (1.480e-2)

0.002

2.000e+4 (0.000e+0)

3.436e+2 (3.376e+2)

0.007

2.000e+4 (0.000e+0)

1.974e-4 (2.009e-4)

0.000

2.000e+4 (0.000e+0)

2.044e-1 (2.601e-1)

0.000

2.000e+4 (0.000e+0)

5.485e-1 (7.929e-1)

-1.054

2.000e+4 (0.000e+0)

6.534e+0 (7.411e-1)

0.003

2.000e+4 (0.000e+0)

7.196e+0 (3.598e+0)

0.142

2.000e+4 (0.000e+0)

1.403e+0 (2.893e-1)

0.917

2.000e+4 (0.000e+0)

2.288e+1 (7.109e+0)

0.093

2.000e+4 (0.000e+0)

4.874e+0 (1.434e+0)

0.002

2.000e+4 (0.000e+0)

1.380e+1 (5.256e+0)

0.079

2.000e+4 (0.000e+0)

1.228e-1 (6.680e-2)

0.008

2.000e+4 (0.000e+0)

4.772e+0 (1.577e+0)

0.286

2.000e+4 (0.000e+0)

3.761e+1 (5.929e+0)

0.267

2.000e+4 (0.000e+0)

1.025e-8 (3.480e-9)

0.000

1.973e+4 (2.490e+2)

1.041e+0 (2.467e-1)

-0.001

2.000e+4 (0.000e+0)

8.190e-9 (1.601e-9)

0.000

8.336e+3 (3.172e+2)

QLPSO

2.576e+1 (1.304e+1)

0.203

2.000e+4 (0.000e+0)

1.034e+0 (7.937e-1)

0.241

2.000e+4 (0.000e+0)

2.166e+4 (4.772e+4)

0.438

2.000e+4 (0.000e+0)

6.309e-1 (1.268e+0)

0.134

2.000e+4 (0.000e+0)

2.837e+5 (6.170e+5)

0.029

2.000e+4 (0.000e+0)

1.774e+1 (1.285e+1)

0.894

2.000e+4 (0.000e+0)

9.990e+0 (1.037e+1)

0.005

2.000e+4 (0.000e+0)

9.063e+1 (8.108e+1)

1.783

2.000e+4 (0.000e+0)

2.026e+0 (6.704e-1)

1.598

2.000e+4 (0.000e+0)

1.020e+2 (5.264e+1)

0.613

2.000e+4 (0.000e+0)

3.362e+1 (7.701e+1)

0.015

2.000e+4 (0.000e+0)

5.207e+1 (2.399e+1)

0.423

2.000e+4 (0.000e+0)

4.461e+0 (3.431e+0)

0.306

2.000e+4 (0.000e+0)

6.817e+0 (5.027e+0)

0.580

2.000e+4 (0.000e+0)

3.508e+1 (1.170e+1)

0.240

2.000e+4 (0.000e+0)

1.794e+1 (1.571e+1)

0.345

1.419e+4 (9.006e+3)

1.705e+0 (3.793e-1)

0.001

2.000e+4 (0.000e+0)

2.508e+0 (3.111e+0)

0.219

2.000e+4 (0.000e+0)

RLEPSO

3.036e+1 (1.313e+1)

0.244

2.000e+4 (4.673e+0)

1.230e+0 (7.697e-1)

0.287

2.000e+4 (4.037e+0)

1.534e+3 (1.591e+3)

0.031

2.000e+4 (4.859e-1)

9.894e-3 (2.018e-2)

0.002

2.000e+4 (1.386e-1)

2.262e+0 (4.908e+0)

0.000

2.000e+4 (0.000e+0)

1.009e+1 (1.097e+1)

0.027

2.000e+4 (1.049e+1)

8.534e+0 (8.118e+0)

0.004

2.000e+4 (0.000e+0)

2.040e+1 (1.295e+1)

0.401

2.000e+4 (7.242e-1)

6.580e-1 (3.068e-1)

0.101

2.000e+4 (7.091e-1)

8.883e+1 (4.011e+1)

0.527

2.000e+4 (1.802e+0)

9.621e+0 (1.213e+1)

0.004

2.000e+4 (0.000e+0)

3.942e+1 (1.897e+1)

0.309

2.000e+4 (3.756e+0)

6.021e+0 (3.136e+0)

0.413

2.000e+4 (1.590e+0)

3.258e+0 (1.908e+0)

0.068

2.000e+4 (5.629e-1)

2.714e+1 (9.355e+0)

0.154

2.000e+4 (5.173e-1)

0.000e+0 (0.000e+0)

0.000

5.373e+3 (2.718e+3)

1.733e+0 (2.791e-1)

0.001

2.003e+4 (2.644e+1)

7.837e-4 (4.890e-3)

0.000

1.425e+4 (6.329e+3)

RL_PSO

7.571e+1 (1.374e+1)

0.648

2.000e+4 (0.000e+0)

2.265e+0 (7.043e-1)

0.528

2.000e+4 (0.000e+0)

2.353e+4 (1.592e+4)

0.476

2.000e+4 (0.000e+0)

1.521e+0 (6.578e-1)

0.323

2.000e+4 (0.000e+0)

1.190e+6 (6.462e+5)

0.121

2.000e+4 (0.000e+0)

1.986e+1 (1.346e+1)

1.135

2.000e+4 (0.000e+0)

5.937e+1 (3.752e+1)

0.027

2.000e+4 (0.000e+0)

5.471e+1 (2.211e+1)

1.076

2.000e+4 (0.000e+0)

2.259e+0 (5.185e-1)

1.852

2.000e+4 (0.000e+0)

1.396e+2 (2.808e+1)

0.861

2.000e+4 (0.000e+0)

1.100e+2 (5.716e+1)

0.049

2.000e+4 (0.000e+0)

8.234e+1 (1.531e+1)

0.696

2.000e+4 (0.000e+0)

1.161e+1 (4.176e+0)

0.796

2.000e+4 (0.000e+0)

1.373e+1 (3.223e+0)

1.573

2.000e+4 (0.000e+0)

6.792e+1 (9.999e+0)

0.596

2.000e+4 (0.000e+0)

5.592e+1 (1.258e+1)

1.076

2.000e+4 (0.000e+0)

2.730e+0 (2.583e-1)

0.004

2.000e+4 (0.000e+0)

2.288e+0 (1.336e+0)

0.199

2.000e+4 (0.000e+0)

DEAP_DE

2.867e+1 (5.111e+0)

0.229

2.000e+4 (0.000e+0)

6.034e-4 (1.729e-3)

0.000

1.999e+4 (6.045e+1)

2.342e+3 (1.353e+3)

0.047

2.000e+4 (0.000e+0)

2.242e-4 (1.044e-4)

0.000

2.000e+4 (0.000e+0)

2.254e+0 (2.369e+0)

0.000

2.000e+4 (0.000e+0)

4.440e+0 (5.340e+0)

-0.613

2.000e+4 (0.000e+0)

7.554e+0 (8.598e-1)

0.003

2.000e+4 (0.000e+0)

2.790e+1 (1.161e+1)

0.549

2.000e+4 (0.000e+0)

1.480e+0 (2.720e-1)

1.001

2.000e+4 (0.000e+0)

3.042e+1 (7.981e+0)

0.142

2.000e+4 (0.000e+0)

6.136e+0 (1.308e+0)

0.003

2.000e+4 (0.000e+0)

2.575e+1 (5.557e+0)

0.186

2.000e+4 (0.000e+0)

2.432e-2 (3.258e-2)

0.002

2.000e+4 (0.000e+0)

9.349e+0 (1.725e+0)

0.944

2.000e+4 (0.000e+0)

4.014e+1 (5.086e+0)

0.295

2.000e+4 (0.000e+0)

0.000e+0 (0.000e+0)

0.000

1.451e+3 (3.768e+2)

1.423e-1 (1.585e-1)

-0.004

1.779e+4 (3.152e+3)

8.305e-9 (1.285e-9)

0.000

4.616e+3 (1.809e+2)

JDE21

1.931e+1 (8.537e+0)

0.145

2.001e+4 (0.000e+0)

2.823e-1 (3.353e-1)

0.066

2.001e+4 (0.000e+0)

7.178e+2 (6.038e+2)

0.015

2.001e+4 (0.000e+0)

3.491e-4 (2.538e-4)

0.000

2.001e+4 (0.000e+0)

1.820e+0 (2.230e+0)

0.000

2.001e+4 (0.000e+0)

1.373e+0 (1.036e+0)

-0.960

2.001e+4 (0.000e+0)

5.697e+0 (2.057e+0)

0.003

2.001e+4 (0.000e+0)

9.422e+0 (5.564e+0)

0.185

2.001e+4 (0.000e+0)

1.499e+0 (2.754e-1)

1.021

2.001e+4 (0.000e+0)

2.697e+1 (1.164e+1)

0.120

2.001e+4 (0.000e+0)

3.293e+0 (2.150e+0)

0.001

2.001e+4 (0.000e+0)

1.740e+1 (8.086e+0)

0.111

2.001e+4 (0.000e+0)

9.219e-1 (1.302e+0)

0.063

2.001e+4 (0.000e+0)

4.948e+0 (2.253e+0)

0.311

2.001e+4 (0.000e+0)

3.782e+1 (7.600e+0)

0.270

2.001e+4 (0.000e+0)

1.492e-8 (2.313e-8)

0.000

1.684e+4 (1.615e+3)

4.500e-1 (2.469e-1)

-0.003

2.001e+4 (0.000e+0)

5.376e-9 (2.571e-9)

-0.000

6.005e+3 (9.560e+2)

MadDE

1.979e+1 (4.538e+0)

0.149

2.000e+4 (0.000e+0)

1.187e-1 (5.727e-2)

0.028

2.000e+4 (0.000e+0)

1.087e+3 (7.153e+2)

0.022

2.000e+4 (0.000e+0)

7.408e-4 (3.416e-4)

0.000

2.000e+4 (0.000e+0)

3.278e+1 (3.199e+1)

0.000

2.000e+4 (0.000e+0)

2.784e-1 (5.658e-1)

-1.084

2.000e+4 (0.000e+0)

2.406e+0 (5.404e-1)

0.001

2.000e+4 (0.000e+0)

1.612e+1 (8.541e+0)

0.317

2.000e+4 (0.000e+0)

1.300e+0 (2.485e-1)

0.804

2.000e+4 (0.000e+0)

2.445e+1 (6.152e+0)

0.103

2.000e+4 (0.000e+0)

2.977e+0 (8.955e-1)

0.001

2.000e+4 (0.000e+0)

1.675e+1 (4.961e+0)

0.105

2.000e+4 (0.000e+0)

9.613e-1 (3.516e-1)

0.066

2.000e+4 (0.000e+0)

2.557e+0 (8.181e-1)

-0.032

2.000e+4 (0.000e+0)

4.152e+1 (5.500e+0)

0.310

2.000e+4 (0.000e+0)

0.000e+0 (0.000e+0)

0.000

5.950e+3 (5.298e+2)

8.503e-1 (1.675e-1)

-0.002

2.000e+4 (0.000e+0)

8.102e-9 (1.456e-9)

0.000

1.946e+4 (1.506e+2)

NL_SHADE_LBC

1.096e+1 (4.495e+0)

0.071

2.000e+4 (0.000e+0)

1.572e-2 (1.181e-2)

0.004

2.000e+4 (0.000e+0)

9.948e+1 (1.017e+2)

0.002

2.000e+4 (0.000e+0)

4.535e-5 (4.049e-5)

0.000

2.000e+4 (0.000e+0)

3.458e-1 (4.376e-1)

0.000

2.000e+4 (0.000e+0)

7.941e-1 (9.031e-1)

-1.026

2.000e+4 (0.000e+0)

6.190e+0 (6.954e-1)

0.003

2.000e+4 (0.000e+0)

2.908e+0 (1.496e+0)

0.057

2.000e+4 (0.000e+0)

1.316e+0 (2.498e-1)

0.821

2.000e+4 (0.000e+0)

1.774e+1 (5.183e+0)

0.059

2.000e+4 (0.000e+0)

4.514e+0 (5.956e-1)

0.002

2.000e+4 (0.000e+0)

1.060e+1 (3.661e+0)

0.050

2.000e+4 (0.000e+0)

9.434e-2 (5.939e-2)

0.006

2.000e+4 (0.000e+0)

3.008e+0 (1.011e+0)

0.032

2.000e+4 (0.000e+0)

3.359e+1 (6.153e+0)

0.224

2.000e+4 (0.000e+0)

0.000e+0 (0.000e+0)

0.000

7.745e+3 (1.428e+3)

1.830e-1 (1.372e-1)

-0.004

2.000e+4 (0.000e+0)

8.113e-9 (1.492e-9)

0.000

1.443e+4 (8.883e+1)

DEAP_PSO

5.734e+1 (8.711e+0)

0.484

2.000e+4 (0.000e+0)

1.616e+0 (2.938e-1)

0.377

2.000e+4 (0.000e+0)

1.001e+4 (4.416e+3)

0.203

2.000e+4 (0.000e+0)

7.851e-1 (2.217e-1)

0.167

2.000e+4 (0.000e+0)

9.655e+5 (3.297e+5)

0.098

2.000e+4 (0.000e+0)

1.195e+1 (8.139e+0)

0.239

2.000e+4 (0.000e+0)

1.557e+2 (4.902e+1)

0.070

2.000e+4 (0.000e+0)

3.135e+1 (1.496e+1)

0.617

2.000e+4 (0.000e+0)

1.538e+0 (2.454e-1)

1.064

2.000e+4 (0.000e+0)

8.095e+1 (1.117e+1)

0.475

2.000e+4 (0.000e+0)

1.560e+2 (7.179e+1)

0.070

2.000e+4 (0.000e+0)

6.183e+1 (8.589e+0)

0.511

2.000e+4 (0.000e+0)

5.863e+0 (1.234e+0)

0.402

2.000e+4 (0.000e+0)

9.250e+0 (1.949e+0)

0.930

2.000e+4 (0.000e+0)

6.097e+1 (7.073e+0)

0.521

2.000e+4 (0.000e+0)

1.479e+0 (4.123e+0)

0.028

2.639e+3 (6.340e+3)

2.701e+0 (2.142e-1)

0.004

2.000e+4 (0.000e+0)

1.832e+0 (7.114e-1)

0.160

2.000e+4 (0.000e+0)

GL_PSO

3.441e+1 (8.340e+0)

0.280

2.000e+4 (0.000e+0)

2.837e-2 (2.982e-2)

0.007

2.000e+4 (0.000e+0)

1.829e+3 (1.487e+3)

0.037

2.000e+4 (0.000e+0)

1.151e-3 (4.663e-4)

0.000

2.000e+4 (0.000e+0)

2.498e+0 (2.031e+0)

0.000

2.000e+4 (0.000e+0)

5.766e+0 (6.615e+0)

-0.462

2.000e+4 (0.000e+0)

7.455e+0 (1.522e+0)

0.003

2.000e+4 (0.000e+0)

1.702e+1 (8.893e+0)

0.335

2.000e+4 (0.000e+0)

1.659e+0 (2.930e-1)

1.196

2.000e+4 (0.000e+0)

2.980e+1 (1.346e+1)

0.138

2.000e+4 (0.000e+0)

5.839e+0 (1.621e+0)

0.003

2.000e+4 (0.000e+0)

2.928e+1 (1.045e+1)

0.218

2.000e+4 (0.000e+0)

2.171e-1 (1.348e-1)

0.015

2.000e+4 (0.000e+0)

5.504e+0 (4.253e+0)

0.391

2.000e+4 (0.000e+0)

4.498e+1 (6.135e+0)

0.347

2.000e+4 (0.000e+0)

4.486e-6 (1.711e-6)

0.000

2.000e+4 (0.000e+0)

1.133e+0 (5.111e-1)

-0.001

2.000e+4 (0.000e+0)

1.336e-6 (8.689e-7)

0.000

2.000e+4 (0.000e+0)

sDMS_PSO

6.169e+1 (1.429e+1)

0.523

2.010e+4 (0.000e+0)

2.183e+0 (5.613e-1)

0.509

2.010e+4 (0.000e+0)

6.201e+3 (4.022e+3)

0.125

2.010e+4 (0.000e+0)

1.164e+0 (6.146e-1)

0.247

2.010e+4 (0.000e+0)

9.706e+5 (5.686e+5)

0.099

2.010e+4 (0.000e+0)

5.611e+0 (4.912e+0)

-0.480

2.010e+4 (0.000e+0)

4.524e+1 (1.789e+1)

0.020

2.010e+4 (0.000e+0)

1.783e+1 (7.667e+0)

0.351

2.010e+4 (0.000e+0)

9.555e-1 (2.326e-1)

0.427

2.010e+4 (0.000e+0)

1.158e+2 (2.857e+1)

0.704

2.010e+4 (0.000e+0)

7.558e+1 (3.751e+1)

0.034

2.010e+4 (0.000e+0)

6.784e+1 (1.259e+1)

0.565

2.010e+4 (0.000e+0)

1.081e+1 (2.992e+0)

0.741

2.010e+4 (0.000e+0)

4.839e+0 (1.572e+0)

0.296

2.010e+4 (0.000e+0)

5.191e+1 (8.943e+0)

0.423

2.010e+4 (0.000e+0)

2.493e+0 (4.591e+0)

0.048

2.010e+4 (0.000e+0)

2.257e+0 (2.692e-1)

0.002

2.010e+4 (0.000e+0)

1.690e+0 (8.260e-1)

0.147

2.010e+4 (0.000e+0)

SAHLPSO

9.973e+1 (1.859e+1)

0.862

2.000e+4 (0.000e+0)

3.037e+0 (1.303e+0)

0.708

2.000e+4 (0.000e+0)

3.714e+4 (3.282e+4)

0.751

2.000e+4 (0.000e+0)

2.458e+0 (1.773e+0)

0.522

2.000e+4 (0.000e+0)

3.918e+6 (3.289e+6)

0.398

2.000e+4 (0.000e+0)

1.618e+1 (1.373e+1)

0.718

2.000e+4 (0.000e+0)

4.799e+2 (3.783e+2)

0.217

2.000e+4 (0.000e+0)

7.927e+1 (3.753e+1)

1.559

2.000e+4 (0.000e+0)

1.499e+0 (5.506e-1)

1.021

2.000e+4 (0.000e+0)

1.302e+2 (5.042e+1)

0.798

2.000e+4 (0.000e+0)

6.969e+2 (6.441e+2)

0.313

2.000e+4 (0.000e+0)

1.016e+2 (2.939e+1)

0.869

2.000e+4 (0.000e+0)

1.196e+1 (6.608e+0)

0.820

2.000e+4 (0.000e+0)

8.623e+0 (4.808e+0)

0.839

2.000e+4 (0.000e+0)

8.963e+1 (1.936e+1)

0.832

2.000e+4 (0.000e+0)

0.000e+0 (0.000e+0)

0.000

1.408e+3 (2.104e+3)

1.898e+1 (3.977e+1)

0.050

2.000e+4 (0.000e+0)

5.000e+0 (2.425e+0)

0.436

2.000e+4 (0.000e+0)

DEAP_CMAES

3.024e+0 (1.276e+0)

0.000

1.978e+4 (1.553e+3)

3.970e-7 (1.643e-6)

0.000

1.267e+4 (3.189e+3)

7.712e-9 (1.585e-9)

0.000

9.546e+3 (3.808e+2)

8.280e-9 (1.261e-9)

0.000

9.175e+3 (3.206e+2)

1.867e-1 (2.996e-1)

0.000

1.989e+4 (4.843e+2)

9.848e+0 (6.766e+0)

0.000

1.878e+4 (4.191e+3)

7.042e-9 (1.896e-9)

0.000

1.375e+4 (5.582e+2)

7.629e-9 (1.903e-9)

0.000

8.462e+3 (2.678e+2)

5.657e-1 (5.858e-1)

0.000

2.000e+4 (0.000e+0)

8.799e+0 (2.061e+0)

0.000

2.000e+4 (0.000e+0)

7.676e-9 (1.797e-9)

0.000

1.400e+4 (6.661e+2)

5.053e+0 (2.635e+0)

0.000

1.977e+4 (1.615e+3)

7.578e-4 (3.335e-3)

0.000

1.512e+4 (3.625e+3)

2.782e+0 (5.052e-1)

0.000

2.000e+4 (0.000e+0)

1.296e+1 (6.876e+0)

0.000

2.000e+4 (0.000e+0)

0.000e+0 (0.000e+0)

0.000

4.549e+2 (5.536e+1)

1.434e+0 (3.287e-1)

0.000

2.000e+4 (0.000e+0)

7.678e-9 (1.619e-9)

0.000

4.760e+3 (1.968e+2)

Random_search

1.153e+2 (1.398e+1)

1.000

2.000e+4 (0.000e+0)

4.290e+0 (5.473e-1)

1.000

2.000e+4 (0.000e+0)

4.943e+4 (1.856e+4)

1.000

2.000e+4 (0.000e+0)

4.706e+0 (1.023e+0)

1.000

2.000e+4 (0.000e+0)

9.844e+6 (2.061e+6)

1.000

2.000e+4 (0.000e+0)

1.867e+1 (5.340e+0)

1.000

2.000e+4 (0.000e+0)

2.216e+3 (8.222e+2)

1.000

2.000e+4 (0.000e+0)

5.083e+1 (1.352e+1)

1.000

2.000e+4 (0.000e+0)

1.480e+0 (2.046e-1)

1.000

2.000e+4 (0.000e+0)

1.608e+2 (2.478e+1)

1.000

2.000e+4 (0.000e+0)

2.230e+3 (7.845e+2)

1.000

2.000e+4 (0.000e+0)

1.162e+2 (1.474e+1)

1.000

2.000e+4 (0.000e+0)

1.458e+1 (2.649e+0)

1.000

2.000e+4 (0.000e+0)

9.740e+0 (1.960e+0)

1.000

2.000e+4 (0.000e+0)

1.051e+2 (1.086e+1)

1.000

2.000e+4 (0.000e+0)

5.197e+1 (6.343e+0)

1.000

2.000e+4 (0.000e+0)

3.503e+2 (2.907e+2)

1.000

2.000e+4 (0.000e+0)

1.147e+1 (2.506e+0)

1.000

2.000e+4 (0.000e+0)

BayesianOptimizer

2.711e+2 (1.512e+0)

2.388

1.000e+2 (2.810e+0)

1.078e+1 (1.964e+0)

2.514

1.000e+2 (1.419e-1)

1.507e+5 (4.680e+0)

3.048

1.000e+2 (4.262e+0)

1.256e+1 (6.513e+0)

2.669

1.000e+2 (1.459e+0)

6.672e+7 (6.914e+0)

6.778

1.000e+2 (1.508e+0)

2.248e+1 (4.781e+0)

1.432

1.000e+2 (1.951e-1)

4.095e+3 (1.702e-1)

1.848

1.000e+2 (7.251e+0)

9.502e+2 (6.814e+0)

18.692

1.000e+2 (9.415e+0)

3.899e+0 (7.184e+0)

3.647

1.000e+2 (2.698e+0)

3.758e+2 (3.164e+0)

2.415

1.000e+2 (5.146e+0)

1.734e+3 (7.584e-1)

0.777

1.000e+2 (1.444e-1)

2.448e+2 (6.146e+0)

2.158

1.000e+2 (9.416e+0)

3.556e+1 (4.824e+0)

2.438

1.000e+2 (6.902e+0)

2.555e+1 (3.146e+0)

3.273

1.000e+2 (4.461e+0)

8.358e+1 (3.165e+0)

0.766

1.000e+2 (9.236e+0)

2.088e+1 (1.335e+0)

0.402

6.800e+1 (4.273e+0)

3.355e+2 (1.827e+0)

0.958

1.000e+2 (6.277e+0)

7.879e-2 (2.723e+0)

0.007

1.000e+2 (1.277e+0)

RNN-OI

3.636e+3 (9.095e-13)

32.363

1.000e+2 (0.000e+0)

1.031e+1 (0.000e+0)

2.404

1.000e+2 (0.000e+0)

8.035e+6 (9.313e-10)

162.551

1.000e+2 (0.000e+0)

3.548e+1 (0.000e+0)

7.539

1.000e+2 (0.000e+0)

1.246e+8 (0.000e+0)

12.658

1.000e+2 (0.000e+0)

7.513e+1 (0.000e+0)

7.396

1.000e+2 (0.000e+0)

2.182e+4 (0.000e+0)

9.848

1.000e+2 (0.000e+0)

8.738e+5 (1.164e-10)

17189.756

1.000e+2 (0.000e+0)

8.855e-1 (0.000e+0)

0.350

1.000e+2 (0.000e+0)

3.256e+3 (4.547e-13)

21.365

1.000e+2 (0.000e+0)

4.056e+5 (5.821e-11)

181.895

1.000e+2 (0.000e+0)

6.452e+2 (0.000e+0)

5.761

1.000e+2 (0.000e+0)

5.695e+1 (7.105e-15)

3.905

1.000e+2 (0.000e+0)

4.929e+1 (0.000e+0)

6.685

1.000e+2 (0.000e+0)

2.866e+2 (0.000e+0)

2.969

1.000e+2 (0.000e+0)

2.386e+2 (5.684e-14)

4.591

1.000e+2 (0.000e+0)

9.987e+4 (0.000e+0)

286.236

1.000e+2 (0.000e+0)

1.373e+2 (0.000e+0)

11.970

1.000e+2 (0.000e+0)