Meta-Learning and Universality: Deep Representations and Gradient Descent can Approximate any Learning Algorithm

Evolutionary principles in self-referential learning

Jurgen Schmidhuber · 1987

Earlier work this paper cites.

Approximation by superpositions of a sigmoidal function

George Cybenko · 1989

Earlier work this paper cites.

On the approximate realization of continuous mappings by neural networks

Ken-Ichi Funahashi · 1989

Earlier work this paper cites.

Multilayer feedforward networks are universal approximators

Kurt Hornik, Maxwell Stinchcombe, and Halbert White · 1989

Earlier work this paper cites.

Universal approximation of an unknown mapping and its derivatives using multilayer feedforward networks

Kurt Hornik, Maxwell Stinchcombe, and Halbert White · 1990

Earlier work this paper cites.

On the optimization of a synaptic learning rule

Samy Bengio, Yoshua Bengio, Jocelyn Cloutier, and Jan Gecsei · 1992

Earlier work this paper cites.

Fast learning for problem classes using knowledge based network initialization

Michael Husken and Christian Goerick · 2000

Earlier work this paper cites.

Learning to learn using gradient descent

Sepp Hochreiter, A Steven Younger, and Peter R Conwell · 2001

Cited alongside, same era.

One shot learning of simple visual concepts

Brenden M Lake, Ruslan Salakhutdinov, Jason Gross, and Joshua B Tenenbaum · 2011

Cited alongside, same era.

Learning to learn by gradient descent by gradient descent

Marcin Andrychowicz, Misha Denil, Sergio Gomez, Matthew W Hoffman, David Pfau, Tom Schaul, and Nando de Freitas · 2016

Cited alongside, same era.

Rl2: Fast reinforcement learning via slow reinforcement learning

Original

Yan Duan, John Schulman, Xi Chen, Peter L Bartlett, Ilya Sutskever, and Pieter Abbeel · 2016

Cited alongside, same era.

Meta-learning with memory-augmented neural networks

Adam Santoro, Sergey Bartunov, Matthew Botvinick, Daan Wierstra, and Timothy Lillicrap · 2016

Cited alongside, same era.

Learning to reinforcement learn

Hypernetworks

David Ha, Andrew Dai, and Quoc V Le · 2017

Closest in time.

Learning to generalize: Meta-learning for domain generalization

Original

Da Li, Yongxin Yang, Yi-Zhe Song, and Timothy M Hospedales · 2017

Closest in time.

Meta networks

Tsendsuren Munkhdalai and Hong Yu · 2017

Closest in time.

Optimization as a model for few-shot learning

Sachin Ravi and Hugo Larochelle · 2017

Closest in time.

Neural network with unbounded activation functions is universal approximator

Sho Sonoda and Noboru Murata · 2017

Closest in time.

A simple neural attentive meta-learner

Nikhil Mishra, Mostafa Rohaninejad, Xi Chen, and Pieter Abbeel · 2018

Closest in time.