Initial Nugget Evaluation Results for the TREC 2024 RAG Track with the AutoNuggetizer Framework

Overview of the TREC 2003 Question Answering Track. In Proceedings of the Twelfth Text REtrieval Conference (TREC 2003) . Gaithersburg, Maryland

Ellen M. Voorhees. 2003 · 2003

Earlier work this paper cites.

Automatically Evaluating Answers to Definition Questions. In Proceedings of the 2005 Human Language Technology Conference and Conference on Empirical Methods in Natural Language Processing (HLT/EMNLP 2005) . Vancouver, British Columbia, Canada

Jimmy Lin and Dina Demner-Fushman. 2005 · 2005

Earlier work this paper cites.

Methods for Automatically Evaluating Answers to Complex Questions

Jimmy Lin and Dina Demner-Fushman. 2006 · 2006

Earlier work this paper cites.

Different Structures for Evaluating Answers to Complex Questions: Pyramids Won’t Topple, and Neither Will Human Assessors. In Proceedings of the 45th Annual Meeting of the Association for Computational Linguistics (ACL 2007) . Prague, Czech Republic

Hoa Trang Dang and Jimmy Lin. 2007 · 2007

Earlier work this paper cites.

Deconstructing Nuggets: The Stability and Reliability of Complex Question Answering Evaluation. In Proceedings of the 30th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2007) . Amsterdam, the Netherlands

Jimmy Lin and Pengyi Zhang. 2007 · 2007

Earlier work this paper cites.

Anserini: Enabling the Use of Lucene for Information Retrieval Research. In Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2017) . Tokyo, Japan

Peilin Yang, Hui Fang, and Jimmy Lin. 2017 · 2017

Earlier work this paper cites.

MS MARCO: A Human Generated MAchine Reading COmprehension Dataset

Original

Payal Bajaj, Daniel Campos, Nick Craswell, Li Deng, Jianfeng Gao, Xiaodong Liu, Rangan Majumder, Andrew McNamara, Bhaskar Mitra, Tri Nguyen, Mir Rosenberg, Xia Song, Alina Stoica, Saurabh Tiwary, and Tong Wang. 2018 · 2018

Cited alongside, same era.

Document Ranking with a Pretrained Sequence-to-Sequence Model. In Findings of the Association for Computational Linguistics: EMNLP 2020 . Association for Computational Linguistics, Online

Rodrigo Nogueira, Zhiying Jiang, Ronak Pradeep, and Jimmy Lin. 2020 · 2020

Cited alongside, same era.

The Expando-Mono-Duo Design Pattern for Text Ranking with Pretrained Sequence-to-Sequence Models

Original

Ronak Pradeep, Rodrigo Nogueira, and Jimmy Lin. 2021 · 2021

Cited alongside, same era.

Overview of the TREC 2022 Deep Learning Track. In Proceedings of the Thirty-First Text REtrieval Conference (TREC 2022) . Gaithersburg, Maryland

Nick Craswell, Bhaskar Mitra, Emine Yilmaz, Daniel Campos, Jimmy Lin, Ellen M. Voorhees, and Ian Soboroff. 2022 · 2022

Cited alongside, same era.

RankZephyr: Effective and Robust Zero-Shot Listwise Reranking is a Breeze!

Original

Ronak Pradeep, Sahel Sharifymoghaddam, and Jimmy Lin. 2023 · 2023

Later among the works it cites.

Arctic-Embed: Scalable, Efficient, and Accurate Text Embedding Models

Original

Luke Merrick, Danmei Xu, Gaurav Nuti, and Daniel Campos. 2024 · 2024

Closest in time.

Ragnarök: A Reusable RAG Framework and Baselines for TREC 2024 Retrieval-Augmented Generation Track

Original

Ronak Pradeep, Nandan Thakur, Sahel Sharifymoghaddam, Eric Zhang, Ryan Nguyen, Daniel Campos, Nick Craswell, and Jimmy Lin. 2024 · 2024

Closest in time.

Researchy Questions: A Dataset of Multi-Perspective, Decompositional Questions for LLM Web Agents

Original

Corby Rosset, Ho-Lam Chung, Guanghui Qin, Ethan C. Chau, Zhuo Feng, Ahmed Awadallah, Jennifer Neville, and Nikhil Rao. 2024 · 2024

Closest in time.