Online Spatio-Temporal Learning in Deep Neural Networks

A Unified Framework of Online Learning Algorithms for Training Recurrent Neural Networks

Original

Marschall, O.; Cho, K.; and Savin, C. 2019 · 1907

Earlier work this paper cites.

A Learning Algorithm for Continually Running Fully Recurrent Neural Networks

Williams, R. J.; and Zipser, D. 1989 · 1989

Earlier work this paper cites.

Neuromorphic electronic systems

Mead, C. 1990 · 1990

Earlier work this paper cites.

Backpropagation through time: what it does and how to do it

Werbos, P. J. 1990 · 1990

Earlier work this paper cites.

TIMIT acoustic phonetic continuous speech corpus

Garofolo, J. S. 1993 · 1993

Earlier work this paper cites.

Building a large annotated corpus of English: The Penn Treebank

Marcus, M.; Marcinkiewicz, M.; and Santorini, B. 1993 · 1993

Earlier work this paper cites.

Gradient-based learning algorithms for recurrent networks and their computational complexity

Williams, R. J.; and Zipser, D. 1995 · 1995

Earlier work this paper cites.

Neural codes: Firing rates and beyond

Gerstner, W.; Kreiter, A.; Markram, H.; and Herz, A. 1997 · 1997

Earlier work this paper cites.

Brain Plasticity and Behavior

Kolb, B.; and Whishaw, I. 1998 · 1998

Earlier work this paper cites.

Gradient-based learning applied to document recognition

Lecun, Y.; Bottou, L.; Bengio, Y.; and Haffner, P. 1998 · 1998

Cited alongside, same era.

Modeling Temporal Dependencies in High-Dimensional Sequences: Application to Polyphonic Music Generation and Transcription

Boulanger-Lewandowski, N.; Bengio, Y.; and Vincent, P. 2012 · 2012

Cited alongside, same era.

LSTM Neural Networks for Language Modeling

Sundermeyer, M.; Schlüter, R.; and Ney, H. 2012 · 2012

Cited alongside, same era.

Speech recognition with deep recurrent neural networks

Graves, A.; Mohamed, A.; and Hinton, G. 2013 · 2013

Cited alongside, same era.

Decoupled Neural Interfaces using Synthetic Gradients

Original

Jaderberg, M.; Czarnecki, W. M.; Osindero, S.; Vinyals, O.; Graves, A.; Silver, D.; and Kavukcuoglu, K. 2016 · 2016

Cited alongside, same era.

Unbiased Online Recurrent Optimization

Tallec, C.; and Ollivier, Y. 2018 · 2018

Later among the works it cites.

SuperSpike: Supervised Learning in Multilayer Spiking Neural Networks

Zenke, F.; and Ganguli, S. 2018 · 2018

Later among the works it cites.

Optimal Kronecker-Sum Approximation of Real Time Recurrent Learning

Benzing, F.; Gauy, M. M.; Mujika, A.; Martinsson, A.; and Steger, A. 2019 · 2019

Later among the works it cites.

Streaming End-to-end Speech Recognition for Mobile Devices

He, Y.; Sainath, T. N.; Prabhavalkar, R.; McGraw, I.; Alvarez, R.; Zhao, D.; Rybach, D.; Kannan, A.; Wu, Y.; Pang, R.; Liang, Q.; Bhatia, D.; Shangguan, Y.; Li, B.; Pundak, G.; Sim, K. C.; Bagby, T.; Chang, S.; Rao, K.; and Gruenstein, A. 2019 · 2019

Later among the works it cites.

Backpropagation through time and the brain

Lillicrap, T. P.; and Santoro, A. 2019 · 2019

Later among the works it cites.

A solution to the learning dilemma for recurrent networks of spiking neurons