Fetching the paper…
Reading the bibliography…
We investigate learning of the online local update rules for neural activations (bodies) and weights (synapses) from scratch.
A learning algorithm for continually running fully recurrent neural networks
Williams, R. J. and Zipser, D · 1989
Earlier work this paper cites.
On the optimization of a synaptic learning rule
Bengio, S., Bengio, Y., Cloutier, J., and Gecsei, J · 1992
Earlier work this paper cites.
Learning to control fast-weight memories: An alternative to dynamic recurrent networks
Schmidhuber, J · 1992
Earlier work this paper cites.
Long short-term memory
Hochreiter, S. and Schmidhuber, J · 1997
Earlier work this paper cites.
Gradient flow in recurrent nets: the difficulty of learning long-term dependencies, 2001
Hochreiter, S., Bengio, Y., Frasconi, P., Schmidhuber, J., et al · 2001
Earlier work this paper cites.
Meta-gradient reinforcement learning
Xu, Z., van Hasselt, H. P., and Silver, D · 2003
Earlier work this paper cites.
Natural evolution strategies
Wierstra, D., Schaul, T., Peters, J., and Schmidhuber, J · 2008
Earlier work this paper cites.
Evolving memory cell structures for sequence learning
Bayer, J., Wierstra, D., Togelius, J., and Schmidhuber, J · 2009
Earlier work this paper cites.
Indirectly encoding neural plasticity as a pattern of local rules
Risi, S. and Stanley, K. O · 2010
Earlier work this paper cites.
Toward nonlinear local reinforcement learning rules through neuroevolution
Vassiliades, V. and Christodoulou, C · 2013
Earlier work this paper cites.
Neural machine translation by jointly learning to align and translate
Bahdanau, D., Cho, K., and Bengio, Y · 2014
Earlier work this paper cites.
Graves, A., Wayne, G., and Danihelka, I · 2014
Earlier work this paper cites.
Sequence to sequence learning with neural networks
Sutskever, I., Vinyals, O., and Le, Q. V · 2014
Earlier work this paper cites.
Weston, J., Chopra, S., and Bordes, A · 2014
Earlier work this paper cites.
Using fast weights to attend to the recent past
Ba, J., Hinton, G. E., Mnih, V., Leibo, J. Z., and Ionescu, C · 2016
Earlier work this paper cites.
Rl2: Fast reinforcement learning via slow reinforcement learning
Duan, Y., Schulman, J., Chen, X., Bartlett, P. L., Sutskever, I., and Abbeel, P · 2016
Earlier work this paper cites.
Convolution by evolution: Differentiable pattern producing networks
Fernando, C., Banarse, D., Reynolds, M., Besse, F., Pfau, D., Jaderberg, M., Lanctot, M., and Wierstra, D · 2016
Earlier work this paper cites.
Ha, D., Dai, A., and Le, Q. V · 2016
Earlier work this paper cites.
Reinforcement learning with unsupervised auxiliary tasks
Jaderberg, M., Mnih, V., Czarnecki, W. M., Schaul, T., Leibo, J. Z., Silver, D., and Kavukcuoglu, K · 2016
Cited alongside, same era.
Learning to learn with backpropagation of hebbian plasticity
Miconi, T · 2016
Cited alongside, same era.
Pixel recurrent neural networks
Oord, A. v. d., Kalchbrenner, N., and Kavukcuoglu, K · 2016
Cited alongside, same era.
The evolution of a generalized neural learning rule
Orchard, J. and Wang, L · 2016
Cited alongside, same era.
Optimization as a model for few-shot learning
Ravi, S. and Larochelle, H · 2016
Cited alongside, same era.
The kanerva machine: A generative distributed memory
Wu, Y., Wayne, G., Graves, A., and Lillicrap, T · 2018
Later among the works it cites.
Dota 2 with large scale deep reinforcement learning
Berner, C., Brockman, G., Chan, B., Cheung, V., Debiak, P., Dennison, C., Farhi, D., Fischer, Q., Hashme, S., Hesse, C., et al · 2019
Later among the works it cites.
Network of evolvable neural units: Evolving to learn at a synaptic level
Bertens, P. and Lee, S.-W · 2019
Later among the works it cites.
Clune, J · 2019
Later among the works it cites.
Transformer-xl: Attentive language models beyond a fixed-length context
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Santoro, A., Bartunov, S., Botvinick, M., Wierstra, D., and Lillicrap, T · 2016
Cited alongside, same era.
Matching networks for one shot learning
Vinyals, O., Blundell, C., Lillicrap, T., Wierstra, D., et al · 2016
Cited alongside, same era.
Learning to reinforcement learn
Wang, J. X., Kurth-Nelson, Z., Tirumala, D., Soyer, H., Leibo, J. Z., Munos, R., Blundell, C., Kumaran, D., and Botvinick, M · 2016
Cited alongside, same era.
Model-agnostic meta-learning for fast adaptation of deep networks
Finn, C., Abbeel, P., and Levine, S · 2017
Cited alongside, same era.
Population based training of neural networks
Jaderberg, M., Dalibard, V., Osindero, S., Czarnecki, W. M., Donahue, J., Razavi, A., Vinyals, O., Green, T., Dunning, I., Simonyan, K., et al · 2017
Cited alongside, same era.
Evolution strategies as a scalable alternative to reinforcement learning
Salimans, T., Ho, J., Chen, X., Sidor, S., and Sutskever, I · 2017
Cited alongside, same era.
Attention is all you need
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, Ł., and Polosukhin, I · 2017
Cited alongside, same era.
Dai, Z., Yang, Z., Yang, Y., Carbonell, J., Le, Q. V., and Salakhutdinov, R · 2019
Later among the works it cites.
Large scale adversarial representation learning
Donahue, J. and Simonyan, K · 2019
Later among the works it cites.
Does learning require memorization? a short tale about a long tail
Feldman, V · 2019
Later among the works it cites.
Shaping belief states with generative environment models for rl
Gregor, K., Rezende, D. J., Besse, F., Wu, Y., Merzic, H., and van den Oord, A · 2019
Later among the works it cites.
Meta-learning biologically plausible semi-supervised update rules
Gu, K., Greydanus, S., Metz, L., Maheswaranathan, N., and Sohl-Dickstein, J · 2019
Later among the works it cites.
Private communication
Hinton, G · 2019
Later among the works it cites.
Optimizing agent behavior over long time scales by transporting value
Hung, C.-C., Lillicrap, T., Abramson, J., Wu, Y., Mirza, M., Carnevale, F., Ahuja, A., and Wayne, G · 2019
Later among the works it cites.
Metalearned neural memory
Munkhdalai, T., Sordoni, A., Wang, T., and Trischler, A · 2019
Later among the works it cites.
Regularized evolution for image classifier architecture search
Real, E., Aggarwal, A., Huang, Y., and Le, Q. V · 2019
Later among the works it cites.
Designing neural networks through neuroevolution
Stanley, K. O., Clune, J., Lehman, J., and Miikkulainen, R · 2019
Later among the works it cites.
Discovery of useful questions as auxiliary tasks
Veeriah, V., Hessel, M., Xu, Z., Rajendran, J., Lewis, R. L., Oh, J., van Hasselt, H. P., Silver, D., and Singh, S · 2019
Later among the works it cites.
Grandmaster level in starcraft ii using multi-agent reinforcement learning
Vinyals, O., Babuschkin, I., Czarnecki, W. M., Mathieu, M., Dudzik, A., Chung, J., Choi, D. H., Powell, R., Ewalds, T., Georgiev, P., et al · 2019
Later among the works it cites.
Backpropamine: training self-modifying neural networks with differentiable neuromodulated plasticity
Miconi, T., Rawal, A., Clune, J., and Stanley, K. O · 2020
Closest in time.