Fetching the paper…

Reading the bibliography…

Longformer for MS MARCO Document Re-ranking Task · Around

2020

Longformer for MS MARCO Document Re-ranking Task

Sekulić, Ivan, Soleimani, Amir, Aliannejadi, Mohammad et al.

Original
PDF
Post on X

Understand

Two step document ranking, where the initial retrieval is done by a classical information retrieval method, followed by neural re-ranking model, is the new standard.

The best performance is achieved by using transformer-based models as re-rankers, e.g., BERT.
We employ Longformer, a BERT-like model for long documents, on the MS MARCO document re-ranking task.
The complete code used for training the model can be found on: https://github.com/isekulic/longformer-marco

Built on

Wolf T, Debut L, Sanh V, Chaumond J, Delangue C, Moi A, Cistac P, Rault T, Louf R, Funtowicz M, Brew J. HuggingFace’s Transformers: State-of-the-art Natural Language Processing. ArXiv. 2019 Oct:arXiv-1910
1910
Earlier work this paper cites.
Kingma DP, Ba J. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980. 2014
Original
2014
Earlier work this paper cites.
Guo J, Fan Y, Ai Q, Croft WB. A deep relevance matching model for ad-hoc retrieval. InProceedings of the 25th ACM International on Conference on Information and Knowledge Management 2016 Oct 24 (pp. 55-64)
2016
Earlier work this paper cites.
Xiong C, Dai Z, Callan J, Liu Z, Power R. End-to-end neural ad-hoc ranking with kernel pooling. InProceedings of the 40th International ACM SIGIR conference on research and development in information retrieval 2017 Aug 7 (pp. 55-64)
2017
Earlier work this paper cites.
Mitra B, Diaz F, Craswell N. Learning to match using local and distributed representations of text for web search. InProceedings of the 26th International Conference on World Wide Web 2017 Apr 3 (pp. 1291-1299)
2017
Earlier work this paper cites.

Similar

Devlin J, Chang MW, Lee K, Toutanova K. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805. 2018 Oct 11
Original
2018
Cited alongside, same era.
Hui K, Yates A, Berberich K, De Melo G. Co-PACRR: A context-aware neural IR model for ad-hoc retrieval. InProceedings of the eleventh ACM international conference on web search and data mining 2018 Feb 2 (pp. 279-287)
2018
Cited alongside, same era.
Nogueira R, Cho K. Passage Re-ranking with BERT. arXiv preprint arXiv:1901.04085. 2019
Original
2019
Cited alongside, same era.
Falcon WE. Pytorch lightning. GitHub. Note: https://github. com/williamFalcon/pytorch-lightning Cited by. 2019
2019
Cited alongside, same era.

Then

Lin J. The neural hype and comparisons against weak baselines. InACM SIGIR Forum 2019 Jan 17 (Vol. 52, No. 2, pp. 40-51). New York, NY, USA: ACM
2019
Later among the works it cites.
Beltagy I, Peters ME, Cohan A. Longformer: The long-document transformer. arXiv preprint arXiv:2004.05150. 2020 Apr 10
Original
2020
Closest in time.
Mitra B, Hofstatter S, Zamani H, Craswell N. Conformer-Kernel with Query Term Independence for Document Retrieval. arXiv preprint arXiv:2007.10434. 2020
Original
2020
Closest in time.
Mitra B, Hofstatter S, Zamani H, Craswell N. Conformer-Kernel with Query Term Independence for Document Retrieval. arXiv preprint arXiv:2007.10434. 2020 Jul 20
Original
2020
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…