Convolutional Tensor-Train LSTM for Spatio-temporal Learning

Understand

Learning from spatio-temporal data has numerous applications such as human-behavior analysis, object tracking, video compression, and physics simulation.However, existing methods still perform poorly on challenging video tasks such as long-term forecasting.

• This is because these kinds of challenging tasks require learning long-term spatio-temporal correlations in the video sequence.
• In this paper, we propose a higher-order convolutional LSTM model that can efficiently learn these correlations, along with a succinct representations of the history.
• This is accomplished through a novel tensor train module that performs prediction by combining convolutional features across time.