A Lipschitz Bandits Approach for Continuous Hyperparameter Optimization

Understand

One of the most critical problems in machine learning is HyperParameter Optimization (HPO), since choice of hyperparameters has a significant impact on final model performance.

• Although there are many HPO algorithms, they either have no theoretical guarantees or require strong assumptions.
• To this end, we introduce BLiE -- a Lipschitz-bandit-based algorithm for HPO that only assumes Lipschitz continuity of the objective function.
• BLiE exploits the landscape of the objective function to adaptively search over the hyperparameter space.