2022

Design-Bench: Benchmarks for Data-Driven Offline Model-Based Optimization

Trabucco, Brandon, Geng, Xinyang, Kumar, Aviral et al.

Understand

Black-box model-based optimization (MBO) problems, where the goal is to find a design input that maximizes an unknown objective function, are ubiquitous in a wide range of domains, such as the design of proteins, DNA sequences, aircraft, and robots.

Solving model-based optimization problems typically requires actively querying the unknown objective function on design proposals, which means physically building the candidate molecule, aircraft, or robot, testing it, and storing the result.
This process can be expensive and time consuming, and one might instead prefer to optimize for the best design using only the data one already has.
This setting -- called offline MBO -- poses substantial and different algorithmic challenges than more commonly studied online techniques.

Built on

Botorch: Programmable bayesian optimization in pytorch
Original
Balandat, M., Karrer, B., Jiang, D. R., Daulton, S., Letham, B., Wilson, A. G., and Bakshy, E · 1910
Earlier work this paper cites.
On the tunability of optimizers in deep learning
Original
Sivaprasad, P. T., Mai, F., Vogels, T., Jaggi, M., and Fleuret, F · 1910
Earlier work this paper cites.
Simulated annealing
Van Laarhoven, P. J. and Aarts, E. H · 1987
Earlier work this paper cites.
Smiles, a chemical language and information system. 1. introduction to methodology and encoding rules
Weininger, D · 1988
Earlier work this paper cites.
Simple statistical gradient-following algorithms for connectionist reinforcement learning
Williams, R. J · 1992
Earlier work this paper cites.
A genetic algorithm tutorial
Whitley, D · 1994
Earlier work this paper cites.
Optimization by direct search: New perspectives on some classical and modern methods
Kolda, T. G., Lewis, R. M., and Torczon, V · 2003
Earlier work this paper cites.
The CMA evolution strategy: A comparing review
Hansen, N · 2006
Earlier work this paper cites.
Evaluating the performance of reinforcement learning algorithms
Original
Jordan, S. M., Chandak, Y., Cohen, D., Zhang, M., and Thomas, P. S · 2006
Earlier work this paper cites.
Gaussian processes for machine learning , volume 2
Williams, C. K. and Rasmussen, C. E · 2006
Earlier work this paper cites.
Practical bayesian optimization
Lizotte, D. J · 2008
Earlier work this paper cites.
Chembl: a large-scale bioactivity database for drug discovery
Gaulton, A., Bellis, L. J., Bento, A. P., Chambers, J., Davies, M., Hersey, A., Light, Y., McGlinchey, S., Michalovich, D., Al-Lazikani, B., and Overington, J. P · 2012
Earlier work this paper cites.
Improving neural networks by preventing co-adaptation of feature detectors
Original
Hinton, G. E., Srivastava, N., Krizhevsky, A., Sutskever, I., and Salakhutdinov, R. R · 2012
Earlier work this paper cites.
Comparative study of firefly algorithm and particle swarm optimization for noisy non-linear optimization problems
Pal, S. K., Rai, C., and Singh, A. P · 2012
Earlier work this paper cites.
Practical bayesian optimization of machine learning algorithms
Snoek, J., Larochelle, H., and Adams, R. P · 2012
Earlier work this paper cites.
Mujoco: A physics engine for model-based control
Todorov, E., Erez, T., and Tassa, Y · 2012
Earlier work this paper cites.
Auto-encoding variational bayes
Original
Kingma, D. P. and Welling, M · 2013
Earlier work this paper cites.

Similar

The cross-entropy method: a unified approach to combinatorial optimization, Monte-Carlo simulation and machine learning
Rubinstein, R. Y. and Kroese, D. P · 2013
Cited alongside, same era.
Generative adversarial nets
Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., and Bengio, Y · 2014
Cited alongside, same era.
Conditional generative adversarial nets
Original
Mirza, M. and Osindero, S · 2014
Cited alongside, same era.
Introduction to the peptide binding problem of computational immunology: New results
Shen, W.-J., Wong, H.-S., Xiao, Q.-W., Guo, X., and Smale, S · 2014
Cited alongside, same era.
Taking the human out of the loop: A review of bayesian optimization
Shahriari, B., Swersky, K., Wang, Z., Adams, R. P., and De Freitas, N · 2015
Cited alongside, same era.
Scalable bayesian optimization using deep neural networks
Snoek, J., Rippel, O., Swersky, K., Kiros, R., Satish, N., Sundaram, N., Patwary, M., Prabhat, M., and Adams, R · 2015
Cited alongside, same era.
Survey of variation in human transcription factors reveals prevalent dna binding changes
Barrera, L. A., Vedenko, A., Kurland, J. V., Rogers, J. M., Gisselbrecht, S. S., Rossin, E. J., Woodard, J., Mariani, L., Kock, K. H., Inukai, S., et al · 2016
Cited alongside, same era.
Openai gym
Original
Brockman, G., Cheung, V., Pettersson, L., Schneider, J., Schulman, J., Tang, J., and Zaremba, W · 2016
Cited alongside, same era.
Local fitness landscape of the green fluorescent protein
Sarkisyan, K. S., Bolotin, D. A., Meer, M. V., Usmanova, D. R., Mishin, A. S., Sharonov, G. V., Ivankov, D. N., Bozhanova, N. G., Baranov, M. S., Soylemez, O., Bogatyreva, N. S., Vlasov, P. K., Egorov, E. S., Logacheva, M. D., Kondrashov, A. S., Chudakov, D. M., Putintseva, E. V., Mamedov, I. Z., Tawfik, D. S., Lukyanov, K. A., and Kondrashov, F. A · 2016
Cited alongside, same era.
Neural architecture search with reinforcement learning
Original
Zoph, B. and Le, Q. V · 2016
Cited alongside, same era.
Self-normalizing neural networks
Klambauer, G., Unterthiner, T., Mayr, A., and Hochreiter, S · 2017
Cited alongside, same era.
Multiple adaptive bayesian linear regression for scalable bayesian optimization with warm start
Original
Perrone, V., Jenatton, R., Seeger, M., and Archambeau, C · 2017
Cited alongside, same era.

Then

Stable baselines
Hill, A., Raffin, A., Ernestus, M., Gleave, A., Kanervisto, A., Traore, R., Dhariwal, P., Hesse, C., Klimov, O., Nichol, A., Plappert, M., Radford, A., Schulman, J., Sidor, S., and Wu, Y · 2018
Later among the works it cites.
ROBEL: RObotics BEnchmarks for Learning with low-cost robots
Ahn, M., Zhu, H., Hartikainen, K., Ponte, H., Gupta, A., Levine, S., and Kumar, V · 2019
Later among the works it cites.
Conditioning by adaptive sampling for robust design
Original
Brookes, D. H., Park, H., and Listgarten, J · 2019
Later among the works it cites.
Bayesian optimization under heavy-tailed payoffs
Chowdhury, S. R. and Gopalan, A · 2019
Later among the works it cites.
Show your work: Improved reporting of experimental results
Dodge, J., Gururangan, S., Card, D., Schwartz, R., and Smith, N. A · 2019
Later among the works it cites.
Model inversion networks for model-based optimization
Original
Kumar, A. and Levine, S · 2019
Later among the works it cites.
Evaluating protein transfer learning with TAPE
Rao, R., Bhattacharya, N., Thomas, N., Duan, Y., Chen, P., Canny, J. F., Abbeel, P., and Song, Y. S · 2019
Later among the works it cites.
Human 5‘ utr design and variant effect prediction from a massively parallel translation assay
Sample, P. J., Wang, B., Reid, D. W., Presnyak, V., McFadyen, I. J., Morris, D. R., and Seelig, G · 2019
Later among the works it cites.
Model-based reinforcement learning for biological sequence design
Angermueller, C., Dohan, D., Belanger, D., Deshpande, R., Murphy, K., and Colwell, L · 2020
Later among the works it cites.
Population-based black-box optimization for biological sequence design
Angermüller, C., Belanger, D., Gane, A., Mariet, Z., Dohan, D., Murphy, K., Colwell, L., and Sculley, D · 2020
Later among the works it cites.
Autofocused oracles for model-based design
Original
Fannjiang, C. and Listgarten, J · 2020
Later among the works it cites.
D4rl: Datasets for deep data-driven reinforcement learning
Original
Fu, J., Kumar, A., Nachum, O., Tucker, G., and Levine, S · 2020
Later among the works it cites.
Firefly algorithm
Yang, X.-S. and Slowik, A · 2020
Later among the works it cites.
Deep reinforcement learning at the edge of the statistical precipice
Agarwal, R., Schwarzer, M., Castro, P. S., Courville, A., and Bellemare, M. G · 2021
Later among the works it cites.
Offline model-based optimization via normalized maximum likelihood estimation
Fu, J. and Levine, S · 2021
Later among the works it cites.
Conservative objective models for effective offline model-based optimization
Trabucco, B., Kumar, A., Geng, X., and Levine, S · 2021
Later among the works it cites.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…