2019

ProBO: Versatile Bayesian Optimization Using Any Probabilistic Programming Language

Neiswanger, Willie, Kandasamy, Kirthevasan, Poczos, Barnabas et al.

Understand

Optimizing an expensive-to-query function is a common task in science and engineering, where it is beneficial to keep the number of queries to a minimum.

A popular strategy is Bayesian optimization (BO), which leverages probabilistic models for this task.
Most BO today uses Gaussian processes (GPs), or a few other surrogate models.
However, there is a broad set of Bayesian modeling techniques that could be used to capture complex systems and reduce the number of queries in BO.

Built on

William R Thompson,
1933
Earlier work this paper cites.
Harold J Kushner,
1964
Earlier work this paper cites.
Jonas Močkus,
1975
Earlier work this paper cites.
Jack W Smith, JE Everhart, WC Dickson, WC Knowler, and RS Johannes,
1988
Earlier work this paper cites.
Bradley Efron,
1992
Earlier work this paper cites.
Peter J Huber,
1992
Earlier work this paper cites.
CL Blake and CJ Merz,
1998
Earlier work this paper cites.
David J Lunn, Andrew Thomas, Nicky Best, and David Spiegelhalter,
2000
Earlier work this paper cites.
Geoffrey E Hinton,
2002
Earlier work this paper cites.
Alexander IJ Forrester, András Sóbester, and Andy J Keane,
2007
Earlier work this paper cites.
Tina Toni, David Welch, Natalja Strelkowa, Andreas Ipsen, and Michael PH Stumpf,
2008
Earlier work this paper cites.
Bianca Schroeder and Garth Gibson,
2009
Earlier work this paper cites.
Niranjan Srinivas, Andreas Krause, Sham M Kakade, and Matthias Seeger,
Original
2009
Earlier work this paper cites.
Matthew D Hoffman and Andrew Gelman,
Original
2011
Earlier work this paper cites.
Frank Hutter, Holger H Hoos, and Kevin Leyton-Brown,
2011
Earlier work this paper cites.
Pasi Jylänki, Jarno Vanhatalo, and Aki Vehtari,
2011
Earlier work this paper cites.
Andreas Krause and Cheng S Ong,
2011
Earlier work this paper cites.
Philipp Hennig and Christian J Schuler,
2012
Earlier work this paper cites.

Similar

Tom Minka,
2012
Cited alongside, same era.
Jasper Snoek, Hugo Larochelle, and Ryan P Adams,
2012
Cited alongside, same era.
Kevin Swersky, Jasper Snoek, and Ryan P Adams,
2012
Cited alongside, same era.
Diederik P Kingma and Max Welling,
Original
2013
Cited alongside, same era.
DD Lucas, R Klein, J Tannahill, D Ivanova, S Brandon, D Domyancic, and Y Zhang,
2013
Cited alongside, same era.
Vikash K Mansinghka, Tejas D Kulkarni, Yura N Perov, and Josh Tenenbaum,
2013
Cited alongside, same era.
S. Ambikasaran, D. Foreman-Mackey, L. Greengard, D. W. Hogg, and M. O’Neil,
2014
Cited alongside, same era.
Jacob R Gardner, Matt J Kusner, Zhixiang Eddie Xu, Kilian Q Weinberger, and John P Cunningham,
2014
Cited alongside, same era.
Michael A Gelbart, Jasper Snoek, and Ryan P Adams,
Original
2014
Cited alongside, same era.
Vikash Mansinghka, Daniel Selsam, and Yura Perov,
Original
2014
Cited alongside, same era.
Willie Neiswanger, Chong Wang, and Eric Xing,
2014
Cited alongside, same era.
Rajesh Ranganath, Sean Gerrish, and David Blei,
2014
Cited alongside, same era.

Then

Tuan Anh Le, Atilim Gunes Baydin, and Frank Wood,
Original
2016
Later among the works it cites.
Tom Rainforth, Tuan Anh Le, Jan-Willem van de Meent, Michael A Osborne, and Frank Wood,
2016
Later among the works it cites.
Daniel Ritchie, Paul Horsfall, and Noah D Goodman,
Original
2016
Later among the works it cites.
John Salvatier, Thomas V Wiecki, and Christopher Fonnesbeck,
2016
Later among the works it cites.
Dustin Tran, Alp Kucukelbir, Adji B. Dieng, Maja Rudolph, Dawen Liang, and David M. Blei,
Original
2016
Later among the works it cites.
Barret Zoph and Quoc V Le,
Original
2016
Later among the works it cites.
Omid Alipourfard, Hongqiang Harry Liu, Jianshu Chen, Shivaram Venkataraman, Minlan Yu, and Ming Zhang,
2017
Later among the works it cites.
Raul Astudillo and P Frazier,
2017
Later among the works it cites.
Rémi Bardenet, Arnaud Doucet, and Chris Holmes,
2017
Later among the works it cites.
Valentin Dalibard, Michael Schaarschmidt, and Eiko Yoneki,
2017
Later among the works it cites.
Joshua V Dillon, Ian Langmore, Dustin Tran, Eugene Brevdo, Srinivas Vasudevan, Dave Moore, Brian Patton, Alex Alemi, Matt Hoffman, and Rif A Saurous,
Original
2017
Later among the works it cites.
Ferenc Huszár,
Original
2017
Later among the works it cites.
Willie Neiswanger and Eric Xing,
2017
Later among the works it cites.
Dustin Tran, Rajesh Ranganath, and David Blei,
2017
Later among the works it cites.
Jian Wu, Matthias Poloczek, Andrew G Wilson, and Peter Frazier,
2017
Later among the works it cites.
Eli Bingham, Jonathan P. Chen, Martin Jankowiak, Fritz Obermeyer, Neeraj Pradhan, Theofanis Karaletsos, Rohit Singh, Paul Szerlip, Paul Horsfall, and Noah D. Goodman,
2018
Later among the works it cites.
Kirthevasan Kandasamy, Willie Neiswanger, Jeff Schneider, Barnabas Poczos, and Eric Xing,
Original
2018
Later among the works it cites.
James T Wilson, Frank Hutter, and Marc Peter Deisenroth,
Original
2018
Later among the works it cites.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…