Deep Sparse Conformer for Speech Recognition

Understand

Conformer has achieved impressive results in Automatic Speech Recognition (ASR) by leveraging transformer's capturing of content-based global interactions and convolutional neural network's exploiting of local features.

• In Conformer, two macaron-like feed-forward layers with half-step residual connections sandwich the multi-head self-attention and convolution modules followed by a post layer normalization.
• We improve Conformer's long-sequence representation ability in two directions, \emph{sparser} and \emph{deeper}.
• We adapt a sparse self-attention mechanism with $\mathcal{O}(L\text{log}L)$ in time complexity and memory usage.