Built on

Fine-grained analysis of optimization and generalization for overparameterized two-layer neural networks
Original
Arora, S · 1901
Earlier work this paper cites.
A generalization theory of gradient descent for learning over-parameterized deep ReLU networks
Original
Cao, Y · 1902
Earlier work this paper cites.
Online meta-learning
Original
Finn, C · 1902
Earlier work this paper cites.
Provable guarantees for gradient-based meta-learning
Original
Khodak, M · 1902
Earlier work this paper cites.
Wide neural networks of any depth evolve as linear models under gradient descent
Original
Lee, J · 1902
Earlier work this paper cites.
A selective overview of deep learning
Original
Fan, J · 1904
Earlier work this paper cites.
Generalization bounds of stochastic gradient descent for wide and deep neural networks
Original
Cao, Y · 1905
Earlier work this paper cites.
Neural proximal/trust region policy optimization attains globally optimal policy
Original
Liu, B · 1906
Earlier work this paper cites.
On the convergence theory of gradient-based model-agnostic meta-learning algorithms
Original
Fallah, A · 1908
Earlier work this paper cites.
Beyond linearization: On quadratic and higher-order approximation of wide neural networks
Original
Bai, Y · 1910
Earlier work this paper cites.
Integrals which are convex functionals
Rockafellar, R · 1968
Earlier work this paper cites.
Learning to predict by the methods of temporal differences
Sutton, R. S · 1988
Earlier work this paper cites.
Convex analysis and variational problems
Ekeland, I · 1999
Earlier work this paper cites.
Approximately optimal approximate reinforcement learning
Kakade, S · 2002
Earlier work this paper cites.
Actor-Critic Algorithms
Konda, V · 2002
Earlier work this paper cites.
Regularized multi–task learning
Evgeniou, T · 2004
Earlier work this paper cites.

Similar

Real and complex analysis
Rudin, W · 2006
Cited alongside, same era.
Finite-dimensional variational inequalities and complementarity problems
Facchinei, F · 2007
Cited alongside, same era.
A survey on transfer learning
Pan, S. J · 2009
Cited alongside, same era.
Transfer learning for reinforcement learning domains: A survey
Taylor, M. E · 2009
Cited alongside, same era.
Learning to learn
Thrun, S · 2012
Cited alongside, same era.
A PAC-Bayesian bound for lifelong learning
Pentina, A · 2014
Cited alongside, same era.
Trust region policy optimization
Schulman, J · 2015
Cited alongside, same era.

Then

Neural tangent kernel: Convergence and generalization in neural networks
Jacot, A · 2018
Later among the works it cites.
Learning overparameterized neural networks via stochastic gradient descent on structured data
Li, Y · 2018
Later among the works it cites.
Deep online learning via meta-learning: Continual adaptation for model-based RL
Original
Nagabandi, A · 2018
Later among the works it cites.
On first-order meta-learning algorithms
Original
Nichol, A · 2018
Later among the works it cites.
Reptile: A scalable meta-learning algorithm
Original
Nichol, A · 2018
Later among the works it cites.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…

2020

On the Global Optimality of Model-Agnostic Meta-Learning

Wang, Lingxiao, Cai, Qi, Yang, Zhuoran et al.

Understand

Model-agnostic meta-learning (MAML) formulates meta-learning as a bilevel optimization problem, where the inner level solves each subtask based on a shared prior, while the outer level searches for the optimal shared prior by optimizing its aggregated performance over all the subtasks.

Despite its empirical success, MAML remains less understood in theory, especially in terms of its global optimality, due to the nonconvexity of the meta-objective (the outer-level objective).
To bridge such a gap between theory and practice, we characterize the optimality gap of the stationary points attained by MAML for both reinforcement learning and supervised learning, where the inner-level and outer-level problems are solved via first-order optimization methods.
In particular, our characterization connects the optimality gap of such stationary points with (i) the functional geometry of inner-level objectives and (ii) the representation power of function approximators, including linear models and neural networks.

Built on

Fine-grained analysis of optimization and generalization for overparameterized two-layer neural networks
Original
Arora, S · 1901
Earlier work this paper cites.
A generalization theory of gradient descent for learning over-parameterized deep ReLU networks
Original
Cao, Y · 1902
Earlier work this paper cites.
Online meta-learning
Original
Finn, C · 1902
Earlier work this paper cites.
Provable guarantees for gradient-based meta-learning
Original
Khodak, M · 1902
Earlier work this paper cites.
Wide neural networks of any depth evolve as linear models under gradient descent
Original
Lee, J · 1902
Earlier work this paper cites.
A selective overview of deep learning
Original
Fan, J · 1904
Earlier work this paper cites.
Generalization bounds of stochastic gradient descent for wide and deep neural networks
Original
Cao, Y · 1905
Earlier work this paper cites.
Neural proximal/trust region policy optimization attains globally optimal policy
Original
Liu, B · 1906
Earlier work this paper cites.
On the convergence theory of gradient-based model-agnostic meta-learning algorithms
Original
Fallah, A · 1908
Earlier work this paper cites.
Beyond linearization: On quadratic and higher-order approximation of wide neural networks
Original
Bai, Y · 1910
Earlier work this paper cites.
Integrals which are convex functionals
Rockafellar, R · 1968
Earlier work this paper cites.
Learning to predict by the methods of temporal differences
Sutton, R. S · 1988
Earlier work this paper cites.
Convex analysis and variational problems
Ekeland, I · 1999
Earlier work this paper cites.
Approximately optimal approximate reinforcement learning
Kakade, S · 2002
Earlier work this paper cites.
Actor-Critic Algorithms
Konda, V · 2002
Earlier work this paper cites.
Regularized multi–task learning
Evgeniou, T · 2004
Earlier work this paper cites.

Similar

Real and complex analysis
Rudin, W · 2006
Cited alongside, same era.
Finite-dimensional variational inequalities and complementarity problems
Facchinei, F · 2007
Cited alongside, same era.
A survey on transfer learning
Pan, S. J · 2009
Cited alongside, same era.
Transfer learning for reinforcement learning domains: A survey
Taylor, M. E · 2009
Cited alongside, same era.
Learning to learn
Thrun, S · 2012
Cited alongside, same era.
A PAC-Bayesian bound for lifelong learning
Pentina, A · 2014
Cited alongside, same era.
Trust region policy optimization
Schulman, J · 2015
Cited alongside, same era.

Then

Neural tangent kernel: Convergence and generalization in neural networks
Jacot, A · 2018
Later among the works it cites.
Learning overparameterized neural networks via stochastic gradient descent on structured data
Li, Y · 2018
Later among the works it cites.
Deep online learning via meta-learning: Continual adaptation for model-based RL
Original
Nagabandi, A · 2018
Later among the works it cites.
On first-order meta-learning algorithms
Original
Nichol, A · 2018
Later among the works it cites.
Reptile: A scalable meta-learning algorithm
Original
Nichol, A · 2018
Later among the works it cites.