On Generalization Bounds of a Family of Recurrent Neural Networks

Understand

Recurrent Neural Networks (RNNs) have been widely applied to sequential data analysis.

• Due to their complicated modeling structures, however, the theory behind is still largely missing.
• To connect theory and practice, we study the generalization properties of vanilla RNNs as well as their variants, including Minimal Gated Unit (MGU), Long Short Term Memory (LSTM), and Convolutional (Conv) RNNs.
• Specifically, our theory is established under the PAC-Learning framework.