2015

Convolutional Neural Network Architectures for Matching Natural Language Sentences

Hu, Baotian, Lu, Zhengdong, Li, Hang et al.

Understand

Semantic matching is of central importance to many natural language tasks \cite{bordes2014semantic,RetrievalQA}.

A successful matching algorithm needs to adequately model the internal structures of language objects and the interaction between them.
As a step toward this goal, we propose convolutional neural network models for matching two sentences, by adapting the convolutional strategy in vision and speech.
The proposed models not only nicely represent the hierarchical structures of sentences with their layer-by-layer composition and pooling, but also capture the rich matching patterns at different levels.

The mathematics of statistical machine translation: Parameter estimation
P. F. Brown, S. A. D. Pietra, V. J. D. Pietra, and R. L. Mercer · 1993
Earlier work this paper cites.
Convolutional networks for images, speech and time series
Y. LeCun and Y. Bengio · 1995
Earlier work this paper cites.
Overfitting in neural nets: Backpropagation, conjugate gradient, and early stopping
C. Rich, L. Steve, and G. Lee · 2000
Earlier work this paper cites.
Rcv1: A new benchmark collection for text categorization research
Y. Lewis, David D.and Yang, T. G. Rose, and F. Li · 2004
Earlier work this paper cites.
Paraphrase identification with lexico-syntactic graph subsumption
V. Rus, P. M. McCarthy, M. C. Lintean, D. S. McNamara, and A. C. Graesser · 2008
Earlier work this paper cites.
Retrieval models for question and answer archives
Original
X. Xue, J. Jiwoon, and C. W. Bruce · 2008
Earlier work this paper cites.
Learning deep architectures for ai
Y. Bengio · 2009
Earlier work this paper cites.
Curriculum learning
Y. Bengio, J. Louradourand, R. Collobert, and J. Weston · 2009
Earlier work this paper cites.
Recurrent neural network based language model
T. Mikolov and M. Karafiát · 2010
Earlier work this paper cites.
Graph kernels
S. V. N. Vishwanathan, N. N. Schraudolph, R. Kondor, and K. M. Borgwardt · 2010
Earlier work this paper cites.

Modeling semantic relevance for question-answer pairs in web social communities
B. Wang, X. Wang, C. Sun, B. Liu, and L. Sun · 2010
Cited alongside, same era.
Natural language processing (almost) from scratch
R. Collobert, J. Weston, L. Bottou, M. Karlen, K. Kavukcuoglu, and P. Kuksa · 2011
Cited alongside, same era.
Dynamic pooling and unfolding recursive autoencoders for paraphrase detection
R. Socher, E. H. Huang, and A. Y. Ng · 2011
Cited alongside, same era.
Parsing Natural Scenes and Natural Language with Recursive Neural Networks
R. Socher, C. C. Lin, A. Y. Ng, and C. D. Manning · 2011
Cited alongside, same era.
Semi-supervised recursive autoencoders for predicting sentiment distributions
R. Socher, J. Pennington, E. H. Huang, A. Y. Ng, and C. D. Manning · 2011
Cited alongside, same era.
Applying convolutional neural networks concepts to hybrid nn-hmm model for speech recognition
O. Abdel-Hamid, A. Mohamed, H. Jiang, and G. Penn · 2012
Cited alongside, same era.
Improving neural networks by preventing co-adaptation of feature detectors
Original
G. E. Hinton, N. Srivastava, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov · 2012
Cited alongside, same era.
Improving deep neural networks for lvcsr using rectified linear units and dropout
G. E. Dahl, T. N. Sainath, and G. E. Hinton · 2013
Cited alongside, same era.
A deep architecture for matching short texts
Z. Lu and H. Li · 2013
Cited alongside, same era.

Efficient estimation of word representations in vector space
Original
T. Mikolov, K. Chen, G. Corrado, and J. Dean · 2013
Later among the works it cites.
Hybrid deep learning for face verification
Y. Sun, X. Wang, and X. Tang · 2013
Later among the works it cites.
A dataset for research on short-text conversations
H. Wang, Z. Lu, H. Li, and E. Chen · 2013
Later among the works it cites.
Learning bilinear model for matching queries and documents
W. Wu, Z. Lu, and H. Li · 2013
Later among the works it cites.
A semantic matching energy function for learning with multi-relational data
B. Antoine, X. Glorot, J. Weston, and Y. Bengio · 2014
Later among the works it cites.
A convolutional neural network for modelling sentences
N. Kalchbrenner, E. Grefenstette, and P. Blunsom · 2014
Later among the works it cites.
Convolutional neural networks for sentence classification
Y. Kim · 2014
Later among the works it cites.
Learning semantic representations using convolutional neural networks for web search
Y. Shen, X. He, J. Gao, L. Deng, and G. Mesnil · 2014
Later among the works it cites.
On dataless hierarchical text classification
Y. Song and D. Roth · 2014
Later among the works it cites.

alphaXiv searches the wider corpus for related work and actual follow-ups.

alphaXiv is searching for related work…

Reading the bibliography…