Beyond backpropagation: bilevel optimization through implicit differentiation and equilibrium propagation

Understand

This paper reviews gradient-based techniques to solve bilevel optimization problems.

• Bilevel optimization is a general way to frame the learning of systems that are implicitly defined through a quantity that they minimize.
• This characterization can be applied to neural networks, optimizers, algorithmic solvers and even physical systems, and allows for greater modeling flexibility compared to an explicit definition of such systems.
• Here we focus on gradient-based approaches that solve such problems.