Fetching the paper…
Reading the bibliography…
In this paper we present Large Language Model Assisted Retrieval Model Ranking (LARMOR), an effective unsupervised approach that leverages LLMs for selecting which dense retriever to use on a test corpus (target).
Predicting query performance. In Proceedings of the 25th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR ’02) . Association for Computing Machinery, 299–306
Steve Cronen-Townsend, Yun Zhou, and W. Bruce Croft. 2002 · 2002
Earlier work this paper cites.
Query performance prediction
Ben He and Iadh Ounis. 2006 · 2006
Earlier work this paper cites.
Query performance prediction in web search environments. In Proceedings of the 30th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR ’07) . Association for Computing Machinery, 543–550
Yun Zhou and W. Bruce Croft. 2007 · 2007
Earlier work this paper cites.
A survey of pre-retrieval query performance predictors. In Proceedings of the 17th ACM Conference on Information and Knowledge Management (CIKM ’08) . 1419–1420
Claudia Hauff, Djoerd Hiemstra, and Franciska de Jong. 2008 · 2008
Earlier work this paper cites.
Reciprocal rank fusion outperforms condorcet and individual rank learning methods. In Proceedings of the 32nd International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR ’09) . ACM, 758–759
Gordon V. Cormack, Charles L. A. Clarke, and Stefan Büttcher. 2009 · 2009
Earlier work this paper cites.
Standard deviation as a query hardness estimator. In Proceedings of the 17th International Conference on String Processing and Information Retrieval (SPIRE’10) . Springer-Verlag, 207–212
Joaquín Pérez-Iglesias and Lourdes Araujo. 2010 · 2010
Earlier work this paper cites.
A similarity measure for indefinite rankings
William Webber, Alistair Moffat, and Justin Zobel. 2010 · 2010
Earlier work this paper cites.
Improved query performance prediction using standard deviation. In Proceedings of the 34th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR ’11) . Association for Computing Machinery, 1089–1090
Ronan Cummins, Joemon Jose, and Colm O’Riordan. 2011 · 2011
Earlier work this paper cites.
Query performance prediction for IR. In Proceedings of the 35th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR ’12) . 1196–1197
David Carmel and Oren Kurland. 2012 · 2012
Earlier work this paper cites.
Back to the roots: a probabilistic framework for query-performance prediction. In Proceedings of the 21st ACM International Conference on Information and Knowledge Management (CIKM ’12) . Association for Computing Machinery, 823–832
Oren Kurland, Anna Shtok, Shay Hummel, Fiana Raiber, David Carmel, and Ofri Rom. 2012 · 2012
Earlier work this paper cites.
Predicting Query Performance by Query-Drift Estimation
Anna Shtok, Oren Kurland, David Carmel, Fiana Raiber, and Gad Markovits. 2012 · 2012
Earlier work this paper cites.
Query Performance Prediction By Considering Score Magnitude and Variance Together. In Proceedings of the 23rd ACM International Conference on Conference on Information and Knowledge Management (CIKM ’14) . Association for Computing Machinery, 1891–1894
Yongquan Tao and Shengli Wu. 2014 · 2014
Earlier work this paper cites.
The effect of pooling and evaluation depth on IR metrics
Xiaolu Lu, Alistair Moffat, and J Shane Culpepper. 2016 · 2016
Earlier work this paper cites.
Query Performance Prediction Using Reference Lists
Anna Shtok, Oren Kurland, and David Carmel. 2016 · 2016
Earlier work this paper cites.
Bert: Pre-training of deep bidirectional transformers for language understanding
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018 · 2018
Earlier work this paper cites.
Fusion in information retrieval. In Proceedings of the 41st International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR ’18) . 1383–1386
Oren Kurland and J Shane Culpepper. 2018 · 2018
Earlier work this paper cites.
Estimating Measurement Uncertainty for Information Retrieval Effectiveness Metrics
Alistair Moffat, Falk Scholer, and Ziying Yang. 2018 · 2018
Earlier work this paper cites.
Roberta: A robustly optimized bert pretraining approach
Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019 · 2019
Earlier work this paper cites.
From doc2query to docTTTTTquery
Rodrigo Nogueira and Jimmy Lin. 2019 · 2019
Earlier work this paper cites.
Language models are few-shot learners. In Proceedings of the 34th International Conference on Neural Information Processing Systems (NIPS ’20) . Curran Associates Inc., Article 159, 25 pages
Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020 · 2020
Earlier work this paper cites.
The Curious Case of Neural Text Degeneration. In Proceedings of the 8th International Conference on Learning Representations (ICLR ’20)
Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. 2020 · 2020
Earlier work this paper cites.
Mandoline: Model Evaluation under Distribution Shift. In Proceedings of the 38th International Conference on Machine Learning (ICML’ 2021)
Mayee Chen, Karan Goel, Nimit Sohoni, Fait Poms, Kayvon Fatahalian, and Christopher Re. 2021 · 2021
Earlier work this paper cites.
Are Labels Always Necessary for Classifier Accuracy Evaluation?. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR ’20)
Weijian Deng and Liang Zheng. 2021 · 2021
Earlier work this paper cites.
Predicting with Confidence on Unseen Distributions. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV ’21) . 1114–1124
Devin Guillory, Vaishaal Shankar, Sayna Ebrahimi, Trevor Darrell, and Ludwig Schmidt. 2021 · 2021
Cited alongside, same era.
A sensitivity analysis of the MSMARCO passage collection
Joel Mackenzie, Matthias Petri, and Alistair Moffat. 2021 · 2021
Cited alongside, same era.
BEIR: A Heterogeneous Benchmark for Zero-shot Evaluation of Information Retrieval Models. In Proceedings of the 35th Conference on Neural Information Processing Systems Datasets and Benchmarks Track (NeurIPS ’21)
Nandan Thakur, Nils Reimers, Andreas Rücklé, Abhishek Srivastava, and Iryna Gurevych. 2021 · 2021
Cited alongside, same era.
GPL: Generative Pseudo Labeling for Unsupervised Domain Adaptation of Dense Retrieval
Kexin Wang, Nandan Thakur, Nils Reimers, and Iryna Gurevych. 2021 · 2021
Cited alongside, same era.
How to Train Your DRAGON: Diverse Augmentation Towards Generalizable Dense Retrieval
Sheng-Chieh Lin, Akari Asai, Minghan Li, Barlas Oguz, Jimmy Lin, Yashar Mehdad, Wen-tau Yih, and Xilun Chen. 2023 · 2023
Later among the works it cites.
Zero-Shot Listwise Document Reranking with a Large Language Model
Xueguang Ma, Xinyu Zhang, Ronak Pradeep, and Jimmy Lin. 2023 · 2023
Later among the works it cites.
Query Performance Prediction: From Ad-hoc to Conversational Search. In Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR ’23) . Association for Computing Machinery, 2583–2593
Chuan Meng, Negar Arabzadeh, Mohammad Aliannejadi, and Maarten de Rijke. 2023 · 2023
Later among the works it cites.
MTEB: Massive Text Embedding Benchmark. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics . Association for Computational Linguistics, 2014–2037
Niklas Muennighoff, Nouamane Tazi, Loic Magne, and Nils Reimers. 2023 · 2023
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Dealing with Typos for BERT-based Passage Retrieval and Ranking. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing (EMNLP ’21) . Association for Computational Linguistics, 2836–2842
Shengyao Zhuang and Guido Zuccon. 2021 · 2021
Cited alongside, same era.
Shallow pooling for sparse labels
Negar Arabzadeh, Alexandra Vtyurina, Xinyi Yan, and Charles L. A. Clarke. 2022 · 2022
Cited alongside, same era.
ranx: A Blazing-Fast Python Library for Ranking Evaluation and Comparison. In Proceedings of the 46th European Conference on Information Retrieval (ECIR) , Vol. 13186. Springer, 259–264
Elias Bassani. 2022 · 2022
Cited alongside, same era.
InPars: Unsupervised Dataset Generation for Information Retrieval. In Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR ’22) . Association for Computing Machinery, 2387–2392
Luiz Bonifacio, Hugo Abonizio, Marzieh Fadaee, and Rodrigo Nogueira. 2022 · 2022
Cited alongside, same era.
Scaling instruction-finetuned language models
Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Yunxuan Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, et al · 2022
Cited alongside, same era.
Promptagator: Few-shot Dense Retrieval From 8 Examples. In Proceedings of the 10th International Conference on Learning Representations (ICLR ’22)
Zhuyun Dai, Vincent Y Zhao, Ji Ma, Yi Luan, Jianmo Ni, Jing Lu, Anton Bakalov, Kelvin Guu, Keith Hall, and Ming-Wei Chang. 2022 · 2022
Cited alongside, same era.
Saurabh Garg, Sivaraman Balakrishnan, Zachary Chase Lipton, Behnam Neyshabur, and Hanie Sedghi. 2022 · 2022
Cited alongside, same era.
MS-Shift: An Analysis of MS MARCO Distribution Shifts on Neural Retrieval. In Proceedings of the 44th European Conference on Information Retrieval (ECIR ’22)
Simon Lupart, Thibault Formal, and Stéphane Clinchant. 2022 · 2022
Cited alongside, same era.
Later among the works it cites.
RankVicuna: Zero-Shot Listwise Document Reranking with Open-Source Large Language Models
Ronak Pradeep, Sahel Sharifymoghaddam, and Jimmy Lin. 2023a · 2023
Later among the works it cites.
RankZephyr: Effective and Robust Zero-Shot Listwise Reranking is a Breeze!
Ronak Pradeep, Sahel Sharifymoghaddam, and Jimmy Lin. 2023b · 2023
Later among the works it cites.
Large language models are effective text rankers with pairwise ranking prompting
Zhen Qin, Rolf Jagerman, Kai Hui, Honglei Zhuang, Junru Wu, Jiaming Shen, Tianqi Liu, Jialu Liu, Donald Metzler, Xuanhui Wang, et al · 2023
Later among the works it cites.
A Thorough Examination on Zero-shot Dense Retrieval. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing (EMNLP ’23) , Houda Bouamor, Juan Pino, and Kalika Bali (Eds.). Association for Computational Linguistics, 15783–15796
Ruiyang Ren, Yingqi Qu, Jing Liu, Xin Zhao, Qifei Wu, Yuchen Ding, Hua Wu, Haifeng Wang, and Ji-Rong Wen. 2023 · 2023
Later among the works it cites.
Is ChatGPT Good at Search? Investigating Large Language Models as Re-Ranking Agents. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing (EMNLP ’23) . Association for Computational Linguistics, 14918–14937
Weiwei Sun, Lingyong Yan, Xinyu Ma, Shuaiqiang Wang, Pengjie Ren, Zhumin Chen, Dawei Yin, and Zhaochun Ren. 2023 · 2023
Later among the works it cites.
Scaling Down, LiTting Up: Efficient Zero-Shot Listwise Reranking with Seq2seq Encoder-Decoder Models
Manveer Singh Tamber, Ronak Pradeep, and Jimmy Lin. 2023 · 2023
Later among the works it cites.
Large language models can accurately predict searcher preferences
Paul Thomas, Seth Spielman, Nick Craswell, and Bhaskar Mitra. 2023 · 2023
Later among the works it cites.
C-Pack: Packaged Resources To Advance General Chinese Embedding
Shitao Xiao, Zheng Liu, Peitian Zhang, and Niklas Muennighoff. 2023 · 2023
Later among the works it cites.
Large language models as optimizers
Chengrun Yang, Xuezhi Wang, Yifeng Lu, Hanxiao Liu, Quoc V Le, Denny Zhou, and Xinyun Chen. 2023 · 2023
Later among the works it cites.
Retrieve anything to augment large language models
Peitian Zhang, Shitao Xiao, Zheng Liu, Zhicheng Dou, and Jian-Yun Nie. 2023 · 2023
Later among the works it cites.
Dense Text Retrieval based on Pretrained Language Models: A Survey
Wayne Xin Zhao, Jing Liu, Ruiyang Ren, and Ji-Rong Wen. 2023a · 2023
Later among the works it cites.
A survey of large language models
Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, et al · 2023
Later among the works it cites.
Beyond yes and no: Improving zero-shot llm rankers via scoring fine-grained relevance labels
Honglei Zhuang, Zhen Qin, Kai Hui, Junru Wu, Le Yan, Xuanhui Wang, and Michael Berdersky. 2023b · 2023
Later among the works it cites.
Open-source Large Language Models are Strong Zero-shot Query Likelihood Models for Document Ranking. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing (EMNLP ’23) . Association for Computational Linguistics, 8807–8817
Shengyao Zhuang, Bing Liu, Bevan Koopman, and Guido Zuccon. 2023a · 2023
Later among the works it cites.
A Setwise Approach for Effective and Highly Efficient Zero-shot Ranking with Large Language Models
Shengyao Zhuang, Honglei Zhuang, Bevan Koopman, and Guido Zuccon. 2023c · 2023
Later among the works it cites.
Llama 3 Model Card
AI@Meta. 2024 · 2024
Closest in time.
Embark on DenseQuest: A System for Selecting the Best Dense Retriever for a Custom Collection. In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR ’24) . Association for Computing Machinery, New York, NY, USA (To Appear)
Ekaterina Khramtsova, Teerapong Leelanupab, Shengyao Zhuang, Mahsa Baktashmotlagh, and Guido Zuccon. 2024 · 2024
Closest in time.
Query Performance Prediction using Relevance Judgments Generated by Large Language Models
Chuan Meng, Negar Arabzadeh, Arian Askari, Mohammad Aliannejadi, and Maarten de Rijke. 2024 · 2024
Closest in time.