Seq-U-Net: A One-Dimensional Causal U-Net for Efficient Sequence Modelling

Understand

Convolutional neural networks (CNNs) with dilated filters such as the Wavenet or the Temporal Convolutional Network (TCN) have shown good results in a variety of sequence modelling tasks.

• However, efficiently modelling long-term dependencies in these sequences is still challenging.
• Although the receptive field of these models grows exponentially with the number of layers, computing the convolutions over very long sequences of features in each layer is time and memory-intensive, prohibiting the use of longer receptive fields in practice.
• To increase efficiency, we make use of the "slow feature" hypothesis stating that many features of interest are slowly varying over time.