Value-Function-based Sequential Minimization for Bi-level Optimization

Understand

Gradient-based Bi-Level Optimization (BLO) methods have been widely applied to handle modern learning tasks.

• However, most existing strategies are theoretically designed based on restrictive assumptions (e.g., convexity of the lower-level sub-problem), and computationally not applicable for high-dimensional tasks.
• Moreover, there are almost no gradient-based methods able to solve BLO in those challenging scenarios, such as BLO with functional constraints and pessimistic BLO.
• In this work, by reformulating BLO into approximated single-level problems, we provide a new algorithm, named Bi-level Value-Function-based Sequential Minimization (BVFSM), to address the above issues.