Fetching the paper…

Reading the bibliography…

2018

Benchmarking Automatic Machine Learning Frameworks

Balaji, Adithya, Allen, Alexander

Original
PDF
Post on X

Understand

AutoML serves as the bridge between varying levels of expertise when designing machine learning systems and expedites the data science process.

A wide range of techniques is taken to address this, however there does not exist an objective comparison of these techniques.
We present a benchmark of current open source AutoML solutions using open source datasets.
We test auto-sklearn, TPOT, auto_ml, and H2O's AutoML solution against a compiled set of regression and classification datasets sourced from OpenML and find that auto-sklearn performs the best across classification datasets and TPOT performs the best across regression datasets.

Benchmarking Automatic Machine Learning Frameworks · Around

Built on

An updated performance comparison of virtual machines and linux containers
Felter, W., Ferreira, A., Rajamony, R., and Rubio, J · 2015
Earlier work this paper cites.
Efficient and robust automated machine learning
Feurer, M., Klein, A., Eggensperger, K., Springenberg, J., Blum, M., and Hutter, F · 2015
Earlier work this paper cites.

Similar

Applications of Evolutionary Computation: 19th European Conference, EvoApplications 2016, Porto, Portugal, March 30 – April 1, 2016, Proceedings, Part I , chapter Automating Biomedical Data Science Through Tree-Based Pipeline Optimization, pp. 123–137
Original
Olson, R. S., Urbanowicz, R. J., Andrews, P. C., Lavender, N. A., Kidd, L. C., and Moore, J. H · 2016
Cited alongside, same era.

Then

Openml benchmarking suites and the openml100, 2017
Bischl, B., Casalicchio, G., Feurer, M., Hutter, F., Lang, M., Mantovani, R. G., van Rijn, J. N., and Vanschoren, J · 2017
Later among the works it cites.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…