Fetching the paper…
Reading the bibliography…
Middle training methods aim to bridge the gap between the Masked Language Model (MLM) pre-training and the final finetuning for retrieval.
Passage Re-ranking with BERT
Rodrigo Nogueira and Kyunghyun Cho. 2019 · 1901
Earlier work this paper cites.
Multiple significance tests: the Bonferroni method
J. M. Bland and D. G. Altman. 1995 · 1995
Earlier work this paper cites.
Improving Efficient Neural Ranking Models with Cross-Architecture Knowledge Distillation
Sebastian Hofstätter, Sophia Althammer, Michael Schröder, Mete Sertkan, and Allan Hanbury. 2020 · 2010
Earlier work this paper cites.
MS MARCO: A human generated machine reading comprehension dataset. In CoCo@ NIPs
Tri Nguyen, Mir Rosenberg, Xia Song, Jianfeng Gao, Saurabh Tiwary, Rangan Majumder, and Li Deng. 2016 · 2016
Earlier work this paper cites.
Attention is all you need. In Advances in Neural Information Processing Systems . 5998–6008
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017 · 2017
Earlier work this paper cites.
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) . Association for Computational Linguistics, Minneapolis, Minnesota, 4171–4186
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019 · 2019
Earlier work this paper cites.
On Losses for Modern Language Models. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) . Association for Computational Linguistics, Online, 4970–4981
Stéphane Aroca-Ouellette and Frank Rudzicz. 2020 · 2020
Earlier work this paper cites.
Pre-training Tasks for Embedding-based Large-scale Retrieval. In International Conference on Learning Representations
Wei-Cheng Chang, Felix X. Yu, Yin-Wen Chang, Yiming Yang, and Sanjiv Kumar. 2020 · 2020
Earlier work this paper cites.
SPLADE: Sparse Lexical and Expansion Model for First Stage Ranking
Thibault Formal, Benjamin Piwowarski, and Stéphane Clinchant. 2021 · 2021
Earlier work this paper cites.
Condenser: a Pre-training Architecture for Dense Retrieval. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing . Association for Computational Linguistics, Online and Punta Cana, Dominican Republic, 981–993
Luyu Gao and Jamie Callan. 2021 · 2021
Earlier work this paper cites.
Efficiently Teaching an Effective Dense Retriever with Balanced Topic Aware Sampling. In Proc. of SIGIR
Sebastian Hofstätter, Sheng-Chieh Lin, Jheng-Hong Yang, Jimmy Lin, and Allan Hanbury. 2021 · 2021
Earlier work this paper cites.
Self-Guided Contrastive Learning for BERT Sentence Representations. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) . Association for Computational Linguistics, Online, 2528–2540
Taeuk Kim, Kang Min Yoo, and Sang-goo Lee. 2021 · 2021
Cited alongside, same era.
In-Batch Negatives for Knowledge Distillation with Tightly-Coupled Teachers for Dense Retrieval. In Proceedings of the 6th Workshop on Representation Learning for NLP (RepL4NLP-2021) . Association for Computational Linguistics, Online, 163–173
Sheng-Chieh Lin, Jheng-Hong Yang, and Jimmy Lin. 2021 · 2021
Cited alongside, same era.
B-PROP: Bootstrapped Pre-training with Representative Words Prediction for Ad-hoc Retrieval
Xinyu Ma, Jiafeng Guo, Ruqing Zhang, Yixing Fan, Xiang Ji, and Xueqi Cheng. 2021b · 2021
Cited alongside, same era.
PROP: Pre-training with Representative Words Prediction for Ad-hoc Retrieval
Xinyu Ma, Jiafeng Guo, Ruqing Zhang, Yixing Fan, Xiang Ji, and Xueqi Cheng. 2021c · 2021
Cited alongside, same era.
Unsupervised Corpus Aware Language Model Pre-training for Dense Passage Retrieval. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) . Association for Computational Linguistics, Dublin, Ireland, 2843–2853
Luyu Gao and Jamie Callan. 2022 · 2022
Later among the works it cites.
Semantic Models for the First-Stage Retrieval: A Comprehensive Review
Jiafeng Guo, Yinqiong Cai, Yixing Fan, Fei Sun, Ruqing Zhang, and Xueqi Cheng. 2022 · 2022
Later among the works it cites.
Establishing Strong Baselines for TripClick Health Retrieval
Sebastian Hofstätter, Sophia Althammer, Mete Sertkan, and Allan Hanbury. 2022 · 2022
Later among the works it cites.
Downstream Datasets Make Surprisingly Good Pretraining Corpora
Kundan Krishna, Saurabh Garg, Jeffrey P. Bigham, and Zachary C. Lipton. 2022 · 2022
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Pre-training for Ad-hoc Retrieval: Hyperlink is Also You Need
Zhengyi Ma, Zhicheng Dou, Wei Xu, Xinyu Zhang, Hao Jiang, Zhao Cao, and Ji rong Wen. 2021a · 2021
Cited alongside, same era.
TripClick: The Log Files of a Large Health Web Search Engine. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval . 2507–2513
Navid Rekabsaz, Oleg Lesota, Markus Schedl, Jon Brassey, and Carsten Eickhoff. 2021 · 2021
Cited alongside, same era.
ColBERTv2: Effective and Efficient Retrieval via Lightweight Late Interaction
Keshav Santhanam, Omar Khattab, Jon Saad-Falcon, Christopher Potts, and Matei Zaharia. 2021 · 2021
Cited alongside, same era.
Are Pretrained Convolutions Better than Pretrained Transformers?. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) . Association for Computational Linguistics, Online, 4349–4359
Yi Tay, Mostafa Dehghani, Jai Prakash Gupta, Vamsi Aribandi, Dara Bahri, Zhen Qin, and Donald Metzler. 2021 · 2021
Cited alongside, same era.
Andrew Yates, Rodrigo Nogueira, and Jimmy Lin. 2021 · 2021
Cited alongside, same era.
ranx: A Blazing-Fast Python Library for Ranking Evaluation and Comparison. In ECIR (2) (Lecture Notes in Computer Science, Vol. 13186) . Springer, 259–264
Elias Bassani. 2022 · 2022
Cited alongside, same era.
From Distillation to Hard Negative Sampling: Making Sparse Neural IR Models More Effective. In Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval (Madrid, Spain) (SIGIR ’22) . Association for Computing Machinery, New York, NY, USA, 2353–2359
Thibault Formal, Carlos Lassance, Benjamin Piwowarski, and Stéphane Clinchant. 2022 · 2022
Cited alongside, same era.
An Experimental Study on Pretraining Transformers from Scratch for IR. In Advances in Information Retrieval , Jaap Kamps, Lorraine Goeuriot, Fabio Crestani, Maria Maistro, Hideo Joho, Brian Davis, Cathal Gurrin, Udo Kruschwitz, and Annalina Caputo (Eds.). Springer Nature Switzerland, Cham, 504–520
Carlos Lassance, Hervé Dejean, and Stéphane Clinchant. 2023a
Cited in the paper.
Carlos Lassance and Stéphane Clinchant. 2022 · 2022
Later among the works it cites.
RetroMAE: Pre-training Retrieval-oriented Transformers via Masked Auto-Encoder
Zheng Liu and Yingxia Shao. 2022 · 2022
Later among the works it cites.
LexMAE: Lexicon-Bottlenecked Pretraining for Large-Scale Retrieval
Tao Shen, Xiubo Geng, Chongyang Tao, Can Xu, Xiaolong Huang, Binxing Jiao, Linjun Yang, and Daxin Jiang. 2022 · 2022
Later among the works it cites.
Scale Efficiently: Insights from Pre-training and Fine-tuning Transformers
Yi Tay, Mostafa Dehghani, Jinfeng Rao, William Fedus, Samira Abnar, Hyung Won Chung, Sharan Narang, Dani Yogatama, Ashish Vaswani, and Donald Metzler. 2022 · 2022
Later among the works it cites.
BEIR: A Heterogeneous Benchmark for Zero-shot Evaluation of Information Retrieval Models
Nandan Thakur, Nils Reimers, Andreas Rücklé, Abhishek Srivastava, and Iryna Gurevych. 2022 · 2022
Later among the works it cites.
Lecture Notes on Neural Information Retrieval
Nicola Tonellotto. 2022 · 2022
Later among the works it cites.
The tale of two MS MARCO – and their unfair comparisons
Carlos Lassance and Stéphane Clinchant. 2023 · 2023
Closest in time.