Molding CNNs for text: non-linear, non-consecutive convolutions

Understand

The success of deep learning often derives from well-chosen operational building blocks.

• In this work, we revise the temporal convolution operation in CNNs to better adapt it to text processing.
• Instead of concatenating word representations, we appeal to tensor algebra and use low-rank n-gram tensors to directly exploit interactions between words already at the convolution stage.
• Moreover, we extend the n-gram convolution to non-consecutive words to recognize patterns with intervening words.