2017

A Deep Relevance Matching Model for Ad-hoc Retrieval

Guo, Jiafeng, Fan, Yixing, Ai, Qingyao et al.

Understand

In recent years, deep neural networks have led to exciting breakthroughs in speech recognition, computer vision, and natural language processing (NLP) tasks.

However, there have been few positive results of deep models on ad-hoc retrieval tasks.
This is partially due to the fact that many important characteristics of the ad-hoc retrieval task have not been well addressed in deep models yet.
Typically, the ad-hoc retrieval task is formalized as a matching problem between two pieces of text in existing work using deep models, and treated equivalent to many NLP tasks such as paraphrase identification, question answering and automatic conversation.

Built on

Learning representations by back-propagating errors
D. R. G. H. R. Williams and G. Hinton · 1986
Earlier work this paper cites.
Viewing morphology as an inference process
R. Krovetz · 1993
Earlier work this paper cites.
Some simple effective approximations to the 2-poisson model for probabilistic weighted retrieval
S. E. Robertson and S. Walker · 1994
Earlier work this paper cites.
Trec and tipster experiments with inquery
J. P. Callan, W. B. Croft, and J. Broglio · 1995
Earlier work this paper cites.
Convolutional networks for images, speech, and time series
Y. LeCun and Y. Bengio · 1995
Earlier work this paper cites.
Overfitting in neural nets: Backpropagation, conjugate gradient, and early stopping
R. C. S. L. L. Giles · 2001
Earlier work this paper cites.
A study of smoothing methods for language models applied to ad hoc information retrieval
C. Zhai and J. Lafferty · 2001
Earlier work this paper cites.
A formal study of information retrieval heuristics
H. Fang, T. Tao, and C. Zhai · 2004
Earlier work this paper cites.
Learning to rank using gradient descent
C. Burges, T. Shaked, E. Renshaw, A. Lazier, M. Deeds, N. Hamilton, and G. Hullender · 2005
Earlier work this paper cites.
Semantic term matching in axiomatic approaches to information retrieval
H. Fang and C. Zhai · 2006
Earlier work this paper cites.
A comparison of statistical significance tests for information retrieval evaluation
M. D. Smucker, J. Allan, and B. Carterette · 2007
Earlier work this paper cites.

Similar

Efficient and effective spam filtering and re-ranking for large web datasets
G. V. Cormack, M. D. Smucker, and C. L. Clarke · 2011
Cited alongside, same era.
Adaptive subgradient methods for online learning and stochastic optimization
J. Duchi, E. Hazan, and Y. Singer · 2011
Cited alongside, same era.
Diagnostic evaluation of information retrieval models
H. Fang, T. Tao, and C. Zhai · 2011
Cited alongside, same era.
Dynamic pooling and unfolding recursive autoencoders for paraphrase detection
R. Socher, E. H. Huang, J. Pennin, C. D. Manning, and A. Y. Ng · 2011
Cited alongside, same era.
Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups
G. Hinton, L. Deng, D. Yu, G. E. Dahl, A.-r. Mohamed, N. Jaitly, A. Senior, V. Vanhoucke, P. Nguyen, T. N. Sainath, et al · 2012
Cited alongside, same era.
Learning deep structured semantic models for web search using clickthrough data
P.-S. Huang, X. He, J. Gao, L. Deng, A. Acero, and L. Heck · 2013
Cited alongside, same era.
A deep architecture for matching short texts
Z. Lu and H. Li · 2013
Cited alongside, same era.
Distributed representations of words and phrases and their compositionality
T. Mikolov, I. Sutskever, K. Chen, G. S. Corrado, and J. Dean · 2013
Cited alongside, same era.
Modeling interestingness with deep neural networks
J. Gao, P. Pantel, M. Gamon, X. He, L. Deng, and Y. Shen · 2014
Cited alongside, same era.
Convolutional neural network architectures for matching natural language sentences
B. Hu, Z. Lu, H. Li, and Q. Chen · 2014
Cited alongside, same era.
A convolutional neural network for modelling sentences
Original
N. Kalchbrenner, E. Grefenstette, and P. Blunsom · 2014
Cited alongside, same era.

Then

Glove: Global vectors for word representation
J. Pennington, R. Socher, and C. D. Manning · 2014
Later among the works it cites.
Learning semantic representations using convolutional neural networks for web search
Y. Shen, X. He, J. Gao, L. Deng, and G. Mesnil · 2014
Later among the works it cites.
Short text similarity with word embeddings
T. Kenter and M. de Rijke · 2015
Later among the works it cites.
Convolutional neural tensor network architecture for community-based question answering
X. Qiu and X. Huang · 2015
Later among the works it cites.
A deep architecture for semantic matching with multiple positional sentence representations
Original
S. Wan, Y. Lan, J. Guo, J. Xu, L. Pang, and X. Cheng · 2015
Later among the works it cites.
Syntax-based deep matching of short texts
Original
M. Wang, Z. Lu, H. Li, and Q. Liu · 2015
Later among the works it cites.
Multigrancnn: An architecture for general matching of text chunks on multiple levels of granularity
W. Yin and H. Schütze · 2015
Later among the works it cites.
Learning to reweight terms with distributed representations
G. Zheng and J. Callan · 2015
Later among the works it cites.
Text matching as image recognition
L. Pang, Y. Lan, J. Guo, J. Xu, S. Wan, and X. Cheng · 2016
Later among the works it cites.
Match-srnn: Modeling the recursive matching structure with spatial rnn
S. Wan, Y. Lan, J. Xu, J. Guo, L. Pang, and X. Cheng · 2016
Later among the works it cites.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…