2017

An Attention-Based Deep Net for Learning to Rank

Wang, Baiyang, Klabjan, Diego

Understand

In information retrieval, learning to rank constructs a machine-based ranking model which given a query, sorts the search results by their degree of relevance or importance to the query.

Neural networks have been successfully applied to this problem, and in this paper, we propose an attention-based deep neural network which better incorporates different embeddings of the queries and search results with an attention-based mechanism.
This model also applies a decoder mechanism to learn the ranks of the search results in a listwise fashion.
The embeddings are trained with convolutional neural networks or the word2vec model.

Built on

Speech understanding systems: summary of results of the five-year research effort at carnegie-mellon university
Reddy, R · 1977
Earlier work this paper cites.
Gradient-based learning applied to document recognition
LeCun, Y., Bottou, L., Bengio, Y., and Haffner, P · 1998
Earlier work this paper cites.
Optimizing search engines using clickthrough data
Joachims, T · 2002
Earlier work this paper cites.
Latent dirichlet allocation
Blei, D., Ng, A., and Jordan, M · 2003
Earlier work this paper cites.
Learning to rank using gradient descent
Burges, C., Shaked, T., Renshaw, E., Lazier, A., Deeds, M., Hamilton, N., and Hullender, G · 2005
Earlier work this paper cites.
Learning to rank with nonsmooth cost functions
Burges, C., Regno, R., and Le, Q · 2006
Earlier work this paper cites.
Learning to rank: from pairwise approach to listwise approach
Cao, Z., Qin, T., Liu, T., Tsai, M., and Li, H · 2007
Earlier work this paper cites.
Frank: a ranking method with fidelity loss
Tsai, M., Liu, T., Qin, T., Chen, H., and Ma, W · 2007
Earlier work this paper cites.
An online algorithm for large scale image similarity learning
Chechik, G., Sharma, V., Shalit, U., and Bengio, S · 2009
Earlier work this paper cites.

Similar

From ranknet to lambdarank to lambdamart: An overview
Burges, C · 2010
Cited alongside, same era.
Convolutional neural network committees for handwritten character classification
Ciresan, D., Meier, ., Gambardella, L., and Schmidhuber, J · 2011
Cited alongside, same era.
Learning to rank for information retrieval
Liu, T · 2011
Cited alongside, same era.
Better digit recognition with a committee of simple neural nets
Meier, U., Ciresan, D., Gambardella, L., and Schmidhuber, J · 2011
Cited alongside, same era.
Mining text data
Aggarwal, C. and Zhai, C. (eds.) · 2012
Cited alongside, same era.
Distributed representations of words and phrases and their compositionality
Mikolov, T., Sutskever, I., Chen, K., Corrado, G., and Dean, J · 2013
Cited alongside, same era.
Recursive deep models for semantic compositionality over a sentiment treebank
Scher, R., Perelygin, A., Wu, J., Chuang, J., Manning, C., Ng, A., and Potts, C · 2013
Cited alongside, same era.
Online multimodal deep similarity learning with application to image retrieval
Wu, P., Hoi, S., Xia, H., Zhao, P., Wang, D., and Miao, C · 2013
Cited alongside, same era.
Glove: Global vectors for word representation
Pennington, J., Socher, R., and Manning, C · 2014
Cited alongside, same era.

Then

Adapting deep ranknet for personalized search
Song, Y., Wang, H., and He, X · 2014
Later among the works it cites.
Dropout: A simple way to prevent neural networks from overfitting
Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., and Salakhutdinov, R · 2014
Later among the works it cites.
Deep learning for content-based image retrieval: a comprehensive study
Wan, J., Wang, D., Hoi, S., Wu, P., Zhu, J., Zhang, Y., and Li, J · 2014
Later among the works it cites.
Learning fine-grained image similarity with deep ranking
Wang, J., Song, Y., Leung, T., Rosenberg, C., Wang, J., Philbin, J., Chen, B., and Wu, Y · 2014
Later among the works it cites.
Neural machine translation by jointly learning to align and translate
Bahdanau, D., Cho, K., and Bengio, Y · 2015
Later among the works it cites.
Describing multimedia content using attention-based encoder-decoder networks
Cho, K., Courville, A., and Bengio, Y · 2015
Later among the works it cites.
Gaussian lda for topic models with word embeddings
Das, R., Zaheer, M., and Dyer, C · 2015
Later among the works it cites.
Learning to rank short text pairs with convolutional deep neural networks
Severyn, A. and Moschitti, A · 2015
Later among the works it cites.
Pointer networks
Vinyals, O., Fortunato, M., and Jaitly, N · 2015
Later among the works it cites.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…