Residual LSTM: Design of a Deep Recurrent Architecture for Distant Speech Recognition

Understand

In this paper, a novel architecture for a deep recurrent neural network, residual LSTM is introduced.

• A plain LSTM has an internal memory cell that can learn long term dependencies of sequential data.
• It also provides a temporal shortcut path to avoid vanishing or exploding gradients in the temporal domain.
• The residual LSTM provides an additional spatial shortcut path from lower layers for efficient training of deep networks with multiple LSTM layers.