$gen$CNN: A Convolutional Architecture for Word Sequence Prediction

Understand

We propose a novel convolutional architecture, named $gen$CNN, for word sequence prediction.

• Different from previous work on neural network-based language modeling and generation (e.g., RNN or LSTM), we choose not to greedily summarize the history of words as a fixed length vector.
• Instead, we use a convolutional neural network to predict the next word with the history of words of variable length.
• Also different from the existing feedforward networks for language modeling, our model can effectively fuse the local correlation and global correlation in the word sequence, with a convolution-gating strategy specifically designed for the task.