Understand
In recent years, an active field of research has developed around automated machine learning (AutoML).
- Unfortunately, comparing different AutoML systems is hard and often done incorrectly.
- We introduce an open, ongoing, and extensible benchmark framework which follows best practices and avoids common mistakes.
- The framework is open-source, uses public datasets and has a website with up-to-date results.
Built on
The weka data mining software: An update
M. Hall, E. Frank, G. Holmes, B. Pfahringer, P. Reutemann, and I.H. Witten · 1931
Earlier work this paper cites.
Scikit-learn: Machine learning in python
F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, et al · 2011
Earlier work this paper cites.
Auto-WEKA: Combined selection and hyperparameter optimization of classification algorithms
C. Thornton, F. Hutter, H. H. Hoos, and K. Leyton-Brown · 2013
Earlier work this paper cites.
OpenML: Networked science in machine learning
J. Vanschoren, J. van Rijn, B. Bischl, and L. Torgo · 2014
Earlier work this paper cites.
Efficient and robust automated machine learning
M. Feurer, A. Klein, K. Eggensperger, J. Springenberg, M. Blum, and F. Hutter · 2015
Earlier work this paper cites.
Similar
A brief review of the chalearn automl challenge: Any-time any-dataset learning without human intervention
G. Isabelle, C. Imad, H.J. Escalante, S. Escalera, D. Jajetic, J.R. Lloyd, N. Macià, B. Ray, l. Romaszko, M. Sebag, A. Statnikov, S. Treguer, and E. Viegas · 2016
Cited alongside, same era.
Efficient hyperparameter optimization and infinitely many armed bandits
L. Li, K.G. Jamieson, G. DeSalvo, A. Rostamizadeh, and A. Talwalkar · 2016
Cited alongside, same era.
Applications of Evolutionary Computation: 19th European Conference, EvoApplications 2016, Porto, Portugal, March 30 – April 1, 2016, Proceedings, Part I , chapter Automating Biomedical Data Science Through Tree-Based Pipeline Optimization, pages 123–137
R.S. Olson, R.J. Urbanowicz, P.C. Andrews, N.A. Lavender, L.C. Kidd, and J.H. Moore · 2016
Cited alongside, same era.
H2O AutoML , August 2017
H2O.ai · 2017
Cited alongside, same era.
B. Bischl, G. Casalicchio, M. Feurer, F. Hutter, M. Lang, R.G. Mantovani, J.N. van Rijn, and J. Vanschoren
Cited in the paper.
Aslib: A benchmark library for algorithm selection
B. Bischl, P. Kerschke, L. Kotthoff, M. Lindauer, Y. Malitsky, A. Frechétte, H. Hoos, F. Hutter, K. Leyton-Brown, K. Tierney, and J. Vanschoren
Cited in the paper.
Then
Benchmarking automatic machine learning frameworks
A. Balaji and A. Allen · 2018
Later among the works it cites.
Ml-plan: Automated machine learning via hierarchical planning
F. Mohr, M. Wever, and E. Hüllermeier · 2018
Later among the works it cites.
Automatic gradient boosting
J. Thomas, S. Coors, and B. Bischl · 2018
Later among the works it cites.
OBOE: collaborative filtering for automl initialization
C. Yang, Y. Akimoto, D.W. Kim, and M. Udell · 2018
Later among the works it cites.
Beyond the bibliography
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…