Fetching the paper…
Reading the bibliography…
Two-level stochastic optimization formulations have become instrumental in a number of machine learning contexts such as continual learning, neural architecture search, adversarial learning, and hyperparameter tuning.
A stochastic approximation method
H. Robbins and S. Monro · 1951
Earlier work this paper cites.
Principles of mathematical analysis
W. Rudin · 1953
Earlier work this paper cites.
On a stochastic approximation method
K. L. Chung · 1954
Earlier work this paper cites.
Asymptotic distribution of stochastic approximation procedures
J. Sacks · 1958
Earlier work this paper cites.
Nonlinear programming: Sequential unconstrained minimization techniques
A. V. Fiacco and G. P. McCormick · 1968
Earlier work this paper cites.
Sensitivity analysis for nonlinear programming using penalty methods
A. V. Fiacco · 1976
Earlier work this paper cites.
Optimality criteria in nonlinear programming
G. P. McCormick · 1976
Earlier work this paper cites.
Introduction to sensitivity and stability analysis in nonlinear programming , volume 165 of Mathematics in Science and Engineering
A. V. Fiacco · 1983
Earlier work this paper cites.
GMRES: A generalized minimal residual algorithm for solving nonsymmetric linear systems
Y. Saad and M. H. Schultz · 1986
Earlier work this paper cites.
Catastrophic interference in connectionist networks: The sequential learning problem
M. McCloskey and N. J. Cohen · 1989
Earlier work this paper cites.
The steepest descent direction for the nonlinear bilevel programming problem
G. Savard and J. Gauvin · 1994
Earlier work this paper cites.
Bilevel and multilevel programming: A bibliography review
L. N. Vicente and P. H. Calamai · 1994
Earlier work this paper cites.
Catastrophic forgetting in connectionist networks
R. M. French · 1999
Earlier work this paper cites.
Foundations of bilevel programming , volume 61 of Nonconvex Optimization and its Applications
S. Dempe · 2002
Earlier work this paper cites.
Numerical Optimization
J. Nocedal and S. J. Wright · 2006
Earlier work this paper cites.
An overview of bilevel optimization
B. Colson, P. Marcotte, and G. Savard · 2007
Earlier work this paper cites.
Bilevel Optimization and Machine Learning , pages 25–47
K. P. Bennett, G. Kunapuli, J. Hu, and J. S. Pang · 2008
Earlier work this paper cites.
Learning multiple layers of features from tiny images
A. Krizhevsky · 2009
Earlier work this paper cites.
Robust stochastic approximation approach to stochastic programming
A. Nemirovski, A. Juditsky, G. Lan, and A. Shapiro · 2009
Earlier work this paper cites.
Practical Bilevel Optimization: Algorithms and Applications
J. F. Bard · 2010
Earlier work this paper cites.
An empirical investigation of catastrophic forgetting in gradient-based neural networks
I. J. Goodfellow, M. Mirza, D. Xiao, A. Courville, and Y. Bengio · 2013
Earlier work this paper cites.
Intriguing properties of neural networks
C. Szegedy, W. Zaremba, I. Sutskever, J. Bruna, D. Erhan, I. Goodfellow, and R. Fergus · 2013
Earlier work this paper cites.
Bi-level stochastic gradient for large scale support vector machine
N. Couellan and W. Wang · 2015
Cited alongside, same era.
Explaining and harnessing adversarial examples
I. Goodfellow, J. Shlens, and C. Szegedy · 2015
Cited alongside, same era.
On the convergence of stochastic bi-level gradient methods
N. Couellan and W. Wang · 2016
Cited alongside, same era.
Hyperparameter optimization with approximate gradient
F. Pedregosa · 2016
Cited alongside, same era.
First-Order Methods in Optimization
A. Beck · 2017
Cited alongside, same era.
Practical Gauss-Newton optimisation for deep learning
A. Botev, H. Ritter, and D. Barber · 2017
Cited alongside, same era.
M. Hong, H. Wai, Z. Wang, and Z. Yang · 2020
Later among the works it cites.
Meta-Learning in neural networks: A survey
T. Hospedales, A. Antoniou, P. Micaelli, and A. Storkey · 2020
Later among the works it cites.
Bilevel continual learning, 2020
Q. Pham, D. Sahoo, C. Liu, and S. C. H. Hoi · 2020
Later among the works it cites.
Bilevel continual learning, 2020
A. Shaker, F. Alesiani, S. Yu, and W. Yin · 2020
Later among the works it cites.
Primal-dual stochastic gradient method for convex programs with many functional constraints
Y. Xu · 2020
Later among the works it cites.
Generalized dataweighting via class-level gradient manipulation
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Weinan E · 2017
Cited alongside, same era.
Gradient episodic memory for continual learning
D. Lopez-Paz and M. Ranzato · 2017
Cited alongside, same era.
Y. Lu, A. Zhong, Q. Li, and B. Dong · 2017
Cited alongside, same era.
Optimization methods for large-scale machine learning
L. Bottou, F. E. Curtis, and J. Nocedal · 2018
Cited alongside, same era.
Bilevel programming for hyperparameter optimization and meta-learning
L. Franceschi, P. Frasconi, S. Salzo, R. Grazzi, and M. Pontil · 2018
Cited alongside, same era.
Approximation Methods for Bilevel Programming
S. Ghadimi and M. Wang · 2018
Cited alongside, same era.
C. Chen, S. Zheng, X. Chen, E. Dong, X. Liu, H. Liu, and D. Dou · 2021
Closest in time.
A Single-Timescale Stochastic Bilevel Optimization Method
T. Chen, Y. Sun, and W. Yin · 2021
Closest in time.
Closing the gap: Tighter analysis of alternating stochastic gradient methods for bilevel problems
T. Chen, Y. Sun, and W. Yin · 2021
Closest in time.
Bilevel optimization: Convergence analysis and enhanced design
K. Ji, J. Yang, and Y. Liang · 2021
Closest in time.
R. Liu, J. Gao, J. Zhang, D. Meng, and Z. Lin · 2021
Closest in time.
A comprehensive survey of neural architecture search: Challenges and solutions
P. Ren, Y. Xiao, X. Chang, P. Huang, Z. Li, X. Chen, and X. Wang · 2021
Closest in time.
Constrained optimization to train neural networks on critical and under-represented Classes
S. Sangalli, E. Erdil, A. Hoetker, O. Donati, and E. Konukoglu · 2021
Closest in time.
On the Convergence Theory for Hessian-Free Bilevel Algorithms
D. Sow, K. Ji, and Y. Liang · 2021
Closest in time.
Provably Faster Algorithms for Bilevel Optimization
J. Yang, K. Ji, and Y. Liang · 2021
Closest in time.
Gradient-based bi-level optimization for deep learning: A survey
C. Chen, X. Chen, C. Ma, Z. Liu, and X. Liu · 2022
Closest in time.
Learning to continuously optimize wireless resource in a dynamic environment: A bilevel optimization perspective
H. Sun, W. Pu, X. Fu, T.H. Chang, and M. Hong · 2022
Closest in time.
An implicit gradient-type method for linearly constrained bilevel problems
I. Tsaknakis, P. Khanduri, and M. Hong · 2022
Closest in time.
Linearly constrained bilevel optimization: A smoothed implicit gradient approach
P. Khanduri, I. Tsaknakis, Y. Zhang, J. Liu, S. Liu, J. Zhang, and M. Hong · 2023
Closest in time.
On Penalty Methods for Nonconvex Bilevel Optimization and First-Order Stochastic Approximation
J. Kwon, D. Kwon, S. Wright, and R. Nowak · 2023
Closest in time.
On Penalty-based Bilevel Gradient Descent Method
H. Shen, Q. Xiao, and T. Chen · 2023
Closest in time.
Alternating projected sgd for equality-constrained bilevel optimization
Q. Xiao, H. Shen, W. Yin, and T. Chen · 2023
Closest in time.