Training recurrent networks online without backtracking

Understand

We introduce the "NoBackTrack" algorithm to train the parameters of dynamical systems such as recurrent neural networks.

• This algorithm works in an online, memoryless setting, thus requiring no backpropagation through time, and is scalable, avoiding the large computational and memory cost of maintaining the full gradient of the current state with respect to the parameters.
• The algorithm essentially maintains, at each time, a single search direction in parameter space.
• The evolution of this search direction is partly stochastic and is constructed in such a way to provide, at every time, an unbiased random estimate of the gradient of the loss function with respect to the parameters.