Towards Scalable Bayesian Optimization via Gradient-Informed Bayesian Neural Networks

Understand

Bayesian optimization (BO) is a widely used method for data-driven optimization that generally relies on zeroth-order data of objective function to construct probabilistic surrogate models.

• These surrogates guide the exploration-exploitation process toward finding global optimum.
• While Gaussian processes (GPs) are commonly employed as surrogates of the unknown objective function, recent studies have highlighted the potential of Bayesian neural networks (BNNs) as scalable and flexible alternatives.
• Moreover, incorporating gradient observations into GPs, when available, has been shown to improve BO performance.