Tensor train decompositions on recurrent networks

Understand

Recurrent neural networks (RNN) such as long-short-term memory (LSTM) networks are essential in a multitude of daily live tasks such as speech, language, video, and multimodal learning.

• The shift from cloud to edge computation intensifies the need to contain the growth of RNN parameters.
• Current research on RNN shows that despite the performance obtained on convolutional neural networks (CNN), keeping a good performance in compressed RNNs is still a challenge.
• Most of the literature on compression focuses on CNNs using matrix product (MPO) operator tensor trains.