Fetching the paper…
Reading the bibliography…
This survey compiles ideas and recommendations from more than a dozen researchers with different backgrounds and from different institutes around the world.
Properties of sufficiency and statistical tests
Maurice Stevenson Bartlett · 1937
Earlier work this paper cites.
The use of ranks to avoid the assumption of normality implicit in the analysis of variance
Milton Friedman · 1937
Earlier work this paper cites.
Individual comparisons by ranking methods
Frank Wilcoxon · 1945
Earlier work this paper cites.
The design of optimum multifactorial experiments
Robin L Plackett and J Peter Burman · 1946
Earlier work this paper cites.
Asymptotic theory of certain” goodness of fit” criteria based on stochastic processes
Theodore W Anderson and Donald A Darling · 1952
Earlier work this paper cites.
Use of ranks in one-criterion variance analysis
William H Kruskal and W Allen Wallis · 1952
Earlier work this paper cites.
The Logic of Scientific Discovery
Karl Raimund Popper · 1959
Earlier work this paper cites.
An Automatic Method for Finding the Greatest or Least Value of a Function
Howard Harry Rosenbrock · 1960
Earlier work this paper cites.
Robust tests for equality of variances
Howard Levene · 1961
Earlier work this paper cites.
A simplex method for function minimization
John A Nelder and Roger Mead · 1965
Earlier work this paper cites.
An analysis of variance test for normality (complete samples)
Samuel Sanford Shapiro and Martin B Wilk · 1965
Earlier work this paper cites.
Conditioning of quasi-newton methods for function minimization
David F Shanno · 1970
Earlier work this paper cites.
Analysis of variance in complex experimental designs
Harold R Lindman · 1974
Earlier work this paper cites.
Objective Knowledge: An Evolutionary Approach
Karl Raimund Popper · 1975
Earlier work this paper cites.
Evolutionsstrategie und numerische Optimierung
Hans-Paul Schwefel · 1975
Earlier work this paper cites.
The Vehicle Routing Problem
Nicos Christofides · 1976
Earlier work this paper cites.
Conjugate gradient methods for indefinite systems
Roger Fletcher · 1976
Earlier work this paper cites.
Exploratory Data Analysis , volume 2
John Wilder Tukey · 1977
Earlier work this paper cites.
On reporting computational experiments with mathematical software
Harlan Crowder, Ron S Dembo, and John M Mulvey · 1979
Earlier work this paper cites.
Testing Unconstrained Optimization Software
Jorge J Moré, Burton S Garbow, and Kenneth E Hillstrom · 1981
Earlier work this paper cites.
Optimization by simulated annealing
Scott Kirkpatrick, C. D. Gelatt, and M. P. Vecchi · 1983
Earlier work this paper cites.
How not to lie with statistics: The correct way to summarize benchmark results
Philip J. Fleming and John J. Wallace · 1986
Earlier work this paper cites.
Experimentation in optimization
Bruce L Golden, Arjang A Assad, Edward A Wasil, and Edward Baker · 1986
Earlier work this paper cites.
Experimental Analysis of Algorithms
Catherine C McGeoch · 1986
Earlier work this paper cites.
Analyzing Simulation Experiments with Common Random Numbers
Jack P. C. Kleijnen · 1988
Earlier work this paper cites.
Tabu search—part i
Fred Glover · 1989
Earlier work this paper cites.
Genetic Algorithms in Search, Optimization, and Machine Learning
David E Goldberg · 1989
Earlier work this paper cites.
Optimization by Simulated Annealing: An Experimental Evaluation. Part I, Graph Partitioning
David S Johnson, Cecilia R Aragon, Lyle A McGeoch, and Catherine Schevon · 1989
Earlier work this paper cites.
Threshold accepting: a general purpose optimization algorithm appearing superior to simulated annealing
Gunter Dueck and Tobias Scheuer · 1990
Earlier work this paper cites.
Optimization by Simulated Annealing: An Experimental Evaluation. Part II, Graph Coloring and Number Partitioning
David S Johnson, Cecilia R Aragon, Lyle A McGeoch, and Catherine Schevon · 1991
Earlier work this paper cites.
Network reoptimization algorithms: A statistically designed comparison
Mohammad M Amini and Richard S Barr · 1993
Earlier work this paper cites.
The Origins of Order: Self-Organization and Selection in Evolution
Stuart A. Kauffman · 1993
Earlier work this paper cites.
Optimal Design of Experiments
F. Pukelsheim · 1993
Earlier work this paper cites.
How not to do it
Ian P. Gent and Toby Walsh · 1994
Earlier work this paper cites.
The hitch-hiker’s guide to evolutionary computation, 1994
Jörg Heitkötter and David Beasley · 1994
Earlier work this paper cites.
Needed: An Empirical Science of Algorithms
John N Hooker · 1994
Earlier work this paper cites.
Designing and reporting on computational experiments with heuristic methods
Richard S Barr, Bruce L Golden, James P Kelly, Mauricio GC Resende, and William R Stewart · 1995
Earlier work this paper cites.
Particle swarm optimization
James Kennedy and Russell Eberhart · 1995
Earlier work this paper cites.
Testing heuristics: We have it all wrong
John N. Hooker · 1996
Earlier work this paper cites.
Toward an experimental method for algorithm simulation
Catherine C McGeoch · 1996
Earlier work this paper cites.
From recombination of genes to the estimation of distributions i. binary parameters
H. Mühlenbein and G. Paaß · 1996
Earlier work this paper cites.
A Robust Solution Searching Scheme in Genetic Search
Shigeyoshi Tsutsui, Ashish Ghosh, and Yoshiji Fujimoto · 1996
Earlier work this paper cites.
Evaluating Evolutionary Algorithms
L. Darrell Whitley, Soraya B. Rana, John Dzubera, and Keith E. Mathias · 1996
Earlier work this paper cites.
Handbook of Evolutionary Computation
Thomas Bäck, David B. Fogel, and Zbigniew Michalewicz · 1997
Earlier work this paper cites.
How not to do it
Ian P. Gent, Stuart A. Grant, Ewen MacIntyre, Patrick Prosser, Paul Shaw, Barbara M. Smith, and Toby Walsh · 1997
Earlier work this paper cites.
Variable neighborhood search
Nenad Mladenović and Pierre Hansen · 1997
Earlier work this paper cites.
Differential Evolution vs. The Functions of the 2
Kenneth V. Price · 1997
Earlier work this paper cites.
No Free Lunch Theorems for Optimization
David H. Wolpert and William G. Macready · 1997
Earlier work this paper cites.
Creating Robust Solutions by Means of Evolutionary Algorithms
Jürgen Branke · 1998
Earlier work this paper cites.
On the futility of blind search: An algorithmic view of “no free lunch”
Joseph C. Culberson · 1998
Earlier work this paper cites.
Efficient global optimization of expensive black-box functions
Donald R. Jones, Matthias Schonlau, and William J. Welch · 1998
Earlier work this paper cites.
A Modified Particle Swarm Optimizer
Yuhui Shi and Russell Eberhart · 1998
Earlier work this paper cites.
CRISP-DM 1.0: Step-by-Step Data Mining Guide
Pete Chapman, Julian Clinton, Randy Kerber, Thomas Khabaza, Thomas Reinartz, Colin Shearer, and Rüdiger Wirth · 2000
Earlier work this paper cites.
Invariance, self-adaptation and correlated mutations in evolution strategies
Nikolaus Hansen · 2000
Earlier work this paper cites.
A Comparison of Three Methods for Selecting Values of Input Variables in the Analysis of Output from a Computer Code
McKay, Michael D and Beckman, Richard J and Conover, William J · 2000
Earlier work this paper cites.
Efficient Fitness Estimation in Noisy Environments
Jürgen Branke, Christian Schmidt, and Hartmut Schmeck · 2001
Earlier work this paper cites.
Mann-whitney test is not just a test of medians: differences in spread can be important
Anna Hart · 2001
Earlier work this paper cites.
Experimental Design for Sensitivity Analysis of Simulation Models
Jack P. C. Kleijnen · 2001
Earlier work this paper cites.
Multivariate permutation tests: with applications in biostatistics , volume 240
Fortunato Pesarin · 2001
Earlier work this paper cites.
Design of experiments using the Taguchi approach: 16 steps to product and process improvement
Ranjit K Roy · 2001
Earlier work this paper cites.
Evolution strategies: A comprehensive introduction
Hans-Georg Beyer and Hans-Paul Schwefel · 2002
Earlier work this paper cites.
A racing algorithm for configuring metaheuristics
Mauro Birattari, Thomas Stützle, Luís Paquete, and Klaus Varrentrapp · 2002
Earlier work this paper cites.
Benchmarking optimization software with performance profiles
Elizabeth D Dolan and Jorge J Moré · 2002
Earlier work this paper cites.
A Critical Note on Experimental Research Methodology in EC
Ágoston Endre Eiben and Márk Jelasity · 2002
Earlier work this paper cites.
Experimental Analysis of Heuristics for the STSP
David Stifler Johnson and Lyle A. McGeoch · 2002
Earlier work this paper cites.
Estimation of Distribution Algorithms: A New Tool for Evolutionary Computation
Pedro Larrañaga and José A. Lozano · 2002
Earlier work this paper cites.
Testing, Evaluation and Performance of Optimization and Learning Systems
L. Darrell Whitley, Jean-Paul Watson, Adele Howe, and Laura Barbulescu · 2002
Earlier work this paper cites.
Classification of metaheuristics and design of experiments for the analysis of components
Mauro Birattari, Luis Paquete, and Thomas Stützle · 2003
Cited alongside, same era.
Reducing the time complexity of the derandomized evolution strategy with covariance matrix adaptation (cma-es)
Nikolaus Hansen, Sibylle D Müller, and Petros Koumoutsakos · 2003
Cited alongside, same era.
The design and analysis of computer experiments , volume 1
Thomas J Santner, Brian J Williams, William I Notz, and Brain J Williams · 2003
Cited alongside, same era.
Handbook of parametric and nonparametric statistical procedures
David J Sheskin · 2003
Cited alongside, same era.
Design and analysis of optimization algorithms using computational statistics
Thomas Bartz-Beielstein, Konstantinos E Parsopoulos, and Michael N Vrahatis · 2004
Cited alongside, same era.
COCO Documentation, Release 15.03, November 2015
Steffen Finck, Nikolaus Hansen, Raymond Ros, and Anne Auger · 2015
Later among the works it cites.
Evolutionary big optimization (BigOpt) of signals
Sim Kuan Goh, Kay Chen Tan, Abdullah Al-Mamun, and Hussein A. Abbass · 2015
Later among the works it cites.
Parameter control in evolutionary algorithms: Trends and challenges
Giorgos Karafotias, Mark Hoogendoorn, and Ágoston E. Eiben · 2015
Later among the works it cites.
Design and Analysis of Simulation Experiments
Jack P. C. Kleijnen · 2015
Later among the works it cites.
Experimentation in Evolutionary Computation , pages 27–54
Mike Preuss · 2015
Later among the works it cites.
Generating new test instances by evolving in instance space
Kate Smith-Miles and Simon Bowly · 2015
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Jialin Liu, Antoine Moreau, Mike Preuss, Baptiste Rozière, Jérémy Rapin, Fabien Teytaud, and Olivier Teytaud · 2004
Cited alongside, same era.
Performance evaluation of an advanced local search evolutionary algorithm
Anne Auger and Nikolaus Hansen · 2005
Cited alongside, same era.
Sequential Parameter Optimization
Thomas Bartz-Beielstein, Christian WG Lasarczyk, and Mike Preuss · 2005
Cited alongside, same era.
Anytime Behavior of Inexact TSP Solvers and Perspectives for Automated Algorithm Selection
Jakob Bossek, Pascal Kerschke, and Heike Trautmann · 2005
Cited alongside, same era.
Evolutionary Optimization in Uncertain Environments – A Survey
Yaochu Jin and Jürgen Branke · 2005
Cited alongside, same era.
Experimental Research in Evolutionary Computation—The New Experimentalism
Thomas Bartz-Beielstein · 2006
Cited alongside, same era.
Ant colony optimization: Artificial ants as a computational intelligence technique
Marco Dorigo, Mauro Birattari, and Thomas Stützle · 2006
Cited alongside, same era.
ASlib: A Benchmark Library for Algorithm Selection
Bernd Bischl, Pascal Kerschke, Lars Kotthoff, Thomas Marius Lindauer, Yuri Malitsky, Alexandre Fréchette, Holger H. Hoos, Frank Hutter, Kevin Leyton-Brown, Kevin Tierney, and Joaquin Vanschoren · 2016
Later among the works it cites.
Noisy Optimization: Fast Convergence Rates with Comparison-based Algorithms
Marie-Liesse Cauwet and Olivier Teytaud · 2016
Later among the works it cites.
Using the shapley value to analyze algorithm portfolios
Alexandre Fréchette, Lars Kotthoff, Tomasz Michalak, Talal Rahwan, Holger H. Hoos, and Kevin Leyton-Brown · 2016
Later among the works it cites.
Requirements for papers focusing on new or improved global optimization algorithms
Raphael T. Haftka · 2016
Later among the works it cites.
Algorithm Engineering: Selected Results and Surveys , volume 9220
Lasse Kliemann and Peter Sanders · 2016
Later among the works it cites.
The irace package: Iterated racing for automatic algorithm configuration
Manuel López-Ibáñez, Jérémie Dubois-Lacoste, Leslie Pérez Cáceres, Thomas Stützle, and Mauro Birattari · 2016
Later among the works it cites.
TPOT: A tree-based pipeline optimization tool for automating machine learning
Randal S Olson and Jason H Moore · 2016
Later among the works it cites.
Low autocorrelation binary sequences
Tom Packebusch and Stephan Mertens · 2016
Later among the works it cites.
COMBO: an efficient Bayesian optimization library for materials science
Tsuyoshi Ueno, Trevor David Rhone, Zhufeng Hou, Teruyasu Mizoguchi, and Koji Tsuda · 2016
Later among the works it cites.
In a nutshell: Sequential parameter optimization
Thomas Bartz-Beielstein, Lorenzo Gentile, and Martin Zaefferer · 2017
Later among the works it cites.
Best Practices for Comparing Optimization Algorithms
Vahid Beiranvand, Warren Hare, and Yves Lucet · 2017
Later among the works it cites.
A Novel Approach to Statistical Comparison of Meta-Heuristic Stochastic Optimization Algorithms Using Deep Statistics
Tome Eftimov, Peter Korošec, and Barbara Koroušić Seljak · 2017
Later among the works it cites.
Regression and Kriging Metamodels with their Experimental Designs in Simulation: A Review
Jack P. C. Kleijnen · 2017
Later among the works it cites.
Hyperband: A Novel Bandit-Based Approach to Hyperparameter Optimization
Lisha Li, Kevin G. Jamieson, Giulia DeSalvo, Afshin Rostamizadeh, and Ameet Talwalkar · 2017
Later among the works it cites.
Design and Analysis of Experiments
Douglas C Montgomery · 2017
Later among the works it cites.
Performance Analysis of Continuous Black-Box Optimization Algorithms via Footprints in Instance Space
Mario A. Muñoz and Kate A. Smith-Miles · 2017
Later among the works it cites.
Pmlb: a large benchmark suite for machine learning evaluation and comparison
Randal S Olson, William La Cava, Patryk Orzechowski, Ryan J Urbanowicz, and Jason H Moore · 2017
Later among the works it cites.
Problem definitions and evaluation criteria for the CEC 2017 competition on constrained real-parameter optimization
Guohua Wu, Rammohan Mallipeddi, and Ponnuthurai Nagaratnam Suganthan · 2017
Later among the works it cites.
IOHprofiler: A Benchmarking and Profiling Tool for Iterative Optimization Heuristics
Carola Doerr, Hao Wang, Furong Ye, Sander van Rijn, and Thomas Bäck · 2018
Later among the works it cites.
The impact of statistics for benchmarking in evolutionary computation research
Tome Eftimov and Peter Korošec · 2018
Later among the works it cites.
The preregistration revolution
Brian A. Nosek, Charles R. Ebersole, Alexander C. DeHaven, and David T. Mellor · 2018
Later among the works it cites.
Where are we now? a large benchmark study of recent symbolic regression methods
Patryk Orzechowski, William La Cava, and Jason H Moore · 2018
Later among the works it cites.
Nevergrad - A gradient-free optimization platform
Jeremy Rapin and Olivier Teytaud · 2018
Later among the works it cites.
Compiling a Benchmarking Test-Suite for Combinatorial Black-Box Optimization: A Position Paper
Ofer M. Shir, Carola Doerr, and Thomas Bäck · 2018
Later among the works it cites.
Difficult features of combinatorial optimization problems and the tunable w-model benchmark problem for simulating them
Thomas Weise and Zijun Wu · 2018
Later among the works it cites.
No Free Lunch Theorem: A Review
Stavros P Adam, Stamatios-Aggelos N Alexandropoulos, Panos M Pardalos, and Michael N Vrahatis · 2019
Later among the works it cites.
Optuna: A next-generation hyperparameter optimization framework
Takuya Akiba, Shotaro Sano, Toshihiko Yanase, Takeru Ohta, and Masanori Koyama · 2019
Later among the works it cites.
Evolutionary Computation for Multicomponent Problems: Opportunities and Future Directions
Mohammad Reza Bonyadi, Zbigniew Michalewicz, Markus Wagner, and Frank Neumann · 2019
Later among the works it cites.
Evolving Diverse TSP Instances by Means of Novel and Creative Mutation Operators
Jakob Bossek, Pascal Kerschke, Aneta Neumann, Markus Wagner, Frank Neumann, and Heike Trautmann · 2019
Later among the works it cites.
Self-adjusting mutation rates with provably optimal success rules
Benjamin Doerr, Carola Doerr, and Johannes Lengler · 2019
Later among the works it cites.
Identifying practical significance through statistical comparison of meta-heuristic stochastic optimization algorithms
Tome Eftimov and Peter Korošec · 2019
Later among the works it cites.
Benchmarking Evolutionary Algorithms For Single-Objective Real-valued Constrained Optimization – A Critical Review
Michael Hellwig and Hans-Georg Beyer · 2019
Later among the works it cites.
Automated Machine Learning: Methods, Systems, Challenges
Frank Hutter, Lars Kotthoff, and Joaquin Vanschoren · 2019
Later among the works it cites.
Automated Algorithm Selection: Survey and Perspectives
Pascal Kerschke, Holger H. Hoos, Frank Neumann, and Heike Trautmann · 2019
Later among the works it cites.
Paradoxes in Numerical Comparison of Optimization Algorithms
Qunfeng Liu, William V. Gehrlein, Ling Wang, Yuan Yan, Yingying Cao, Wei Chen, and Yun Li · 2019
Later among the works it cites.
Evolutionary diversity optimization using multi-objective indicators
Aneta Neumann, Wanru Gao, Markus Wagner, and Frank Neumann · 2019
Later among the works it cites.
Mixed-integer benchmark problems for single- and bi-objective optimization
Tea Tušar, Dimo Brockhoff, and Nikolaus Hansen · 2019
Later among the works it cites.
Single- and Multi-Objective Game-Benchmark for Evolutionary Algorithms
Vanessa Volz, Boris Naujoks, Pascal Kerschke, and Tea Tušar · 2019
Later among the works it cites.
Fireworks algorithm versus plant propagation algorithm
Wouter Vrielink and Daan van den Berg · 2019
Later among the works it cites.
jsspinstancesandresults: Results, data, and instances of the job shop scheduling problem, 2019
Thomas Weise · 2019
Later among the works it cites.
Better software analytics via “duo”: Data mining algorithms using/used-by optimizers
Amritanshu Agrawal, Tim Menzies, Leandro L. Minku, Markus Wagner, and Zhe Yu · 2020
Closest in time.
Benchmarking in Optimization: Best Practice and Open Issues
Thomas Bartz-Beielstein, Carola Doerr, Jakob Bossek, Sowmya Chandrasekaran, Tome Eftimov, Andreas Fischbach, Pascal Kerschke, Manuel Lopez-Ibanez, Katherine M. Malan, Jason H. Moore, Boris Naujoks, Patryk Orzechowski, Vanessa Volz, Markus Wagner, and Thomas Weise · 2020
Closest in time.
Black-box mixed-variable optimisation using a surrogate model that satisfies integer constraints
Laurens Bliek, Sicco Verwer, and Mathijs de Weerdt · 2020
Closest in time.
Towards rigorous validation of energy optimisation experiments
Mahmoud A. Bokhari, Brad Alexander, and Markus Wagner · 2020
Closest in time.
Parameter sensitivity patterns in the plant propagation algorithm
Marleen De Jonge and Daan van den Berg · 2020
Closest in time.
A benchmark suite for designing combinational logic circuits via metaheuristics
Lucas Augusto Müller de Souza, José Eduardo Henriques da Silva, Luciano Jerez Chaves, and Heder Soares Bernardino · 2020
Closest in time.
Theory of Evolutionary Computation – Recent Developments in Discrete Optimization
Benjamin Doerr and Frank Neumann · 2020
Closest in time.
Benchmarking Discrete Optimization Heuristics with IOHprofiler
Carola Doerr, Furong Ye, Naama Horesh, Hao Wang, Ofer M. Shir, and Thomas Bäck · 2020
Closest in time.
Dsctool: A web-service-based framework for statistical comparison of stochastic optimization algorithms
Tome Eftimov, Gašper Petelin, and Peter Korošec · 2020
Closest in time.
Improving the reliability of test functions generators
Andreas Fischbach and Thomas Bartz-Beielstein · 2020
Closest in time.
CMA-ES/pycma: r3.0.3, 2020
Nikolaus Hansen, Youhei Akimoto, yoshihikoueno, Dimo Brockhoff, and Matthew Chan · 2020
Closest in time.
Algorithm Selection of Anytime Algorithms
Alexandre D. Jesus, Arnaud Liefooghe, Bilel Derbel, and Luís Paquete · 2020
Closest in time.
Guidelines for real-world single-objective constrained optimisation competition
Abhishek Kumar, Guohua Wu, Mostafa Z. Ali, Rammohan Mallipeddi, Ponnuthurai Nagaratnam Suganthan, and Swagatam Das · 2020
Closest in time.
When and why metaheuristics researchers can ignore ”no free lunch” theorems
James McDermott · 2020
Closest in time.
Benchmarking manifold learning methods on a large collection of datasets
Patryk Orzechowski, Franciszek Magiera, and Jason H Moore · 2020
Closest in time.
The dynamic travelling thief problem: Benchmarks and performance of evolutionary algorithms, 2020
Ragav Sachdeva, Frank Neumann, and Markus Wagner · 2020
Closest in time.
Understanding the Problem Space in Single-Objective Numerical Optimization Using Exploratory Landscape Analysis
Urban Škvorc, Tome Eftimov, and Peter Korošec · 2020
Closest in time.
A new taxonomy of global optimization algorithms, Nov 2020
Jörg Stork, A. E. Eiben, and Thomas Bartz-Beielstein · 2020
Closest in time.
An Easy-to-Use Real-World Multi-Objective Optimization Problem Suite
Ryoji Tanabe and Hisao Ishibuchi · 2020
Closest in time.
Research trends in combinatorial optimisation
Jann Michael Weinand, Kenneth Sörensen, Pablo San Segundo, Max Kleinebrahm, and Russell McKenna · 2020
Closest in time.
Towards improved benchmarking of black-box optimization algorithms using clustering problems
Marcus Gallagher · 2094
Closest in time.