LongBench v2: Towards Deeper Understanding and Reasoning on Realistic Long-context Multitasks

Wikidata: a free collaborative knowledgebase

Denny Vrandečić and Markus Krötzsch. 2014 · 2014

Earlier work this paper cites.

Why we need new evaluation metrics for nlg

Jekaterina Novikova, Ondřej Dušek, Amanda Cercas Curry, and Verena Rieser. 2017 · 2017

Earlier work this paper cites.

The narrativeqa reading comprehension challenge

Tomáš Kočiskỳ, Jonathan Schwarz, Phil Blunsom, Chris Dyer, Karl Moritz Hermann, Gábor Melis, and Edward Grefenstette. 2018 · 2018

Earlier work this paper cites.

Drop: A reading comprehension benchmark requiring discrete reasoning over paragraphs

Dheeru Dua, Yizhong Wang, Pradeep Dasigi, Gabriel Stanovsky, Sameer Singh, and Matt Gardner. 2019 · 2019

Earlier work this paper cites.

Docred: A large-scale document-level relation extraction dataset

Yuan Yao, Deming Ye, Peng Li, Xu Han, Yankai Lin, Zhenghao Liu, Zhiyuan Liu, Lixin Huang, Jie Zhou, and Maosong Sun. 2019 · 2019

Earlier work this paper cites.

Goemotions: A dataset of fine-grained emotions

Dorottya Demszky, Dana Movshovitz-Attias, Jeongwoo Ko, Alan Cowen, Gaurav Nemade, and Sujith Ravi. 2020 · 2020

Earlier work this paper cites.

Retrieval-augmented generation for knowledge-intensive nlp tasks

Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, et al. 2020 · 2020

Earlier work this paper cites.

Maven: A massive general domain event detection dataset

Xiaozhi Wang, Ziqi Wang, Xu Han, Wangyi Jiang, Rong Han, Zhiyuan Liu, Juanzi Li, Peng Li, Yankai Lin, and Jie Zhou. 2020 · 2020

Earlier work this paper cites.

A dataset of information-seeking questions and answers anchored in research papers

Pradeep Dasigi, Kyle Lo, Iz Beltagy, Arman Cohan, Noah A Smith, and Matt Gardner. 2021 · 2021

Earlier work this paper cites.

Few-nerd: A few-shot named entity recognition dataset

Ning Ding, Guangwei Xu, Yulin Chen, Xiaobin Wang, Xu Han, Pengjun Xie, Haitao Zheng, and Zhiyuan Liu. 2021 · 2021

Earlier work this paper cites.

Efficient attentions for long document summarization

Luyang Huang, Shuyang Cao, Nikolaus Parulian, Heng Ji, and Lu Wang. 2021 · 2021

Earlier work this paper cites.

Qmsum: A new benchmark for query-based multi-domain meeting summarization

Ming Zhong, Da Yin, Tao Yu, Ahmad Zaidi, Mutethia Mutuma, Rahul Jha, Ahmed Hassan, Asli Celikyilmaz, Yang Liu, Xipeng Qiu, et al. 2021 · 2021

Earlier work this paper cites.

Kqa pro: A dataset with explicit compositional programs for complex question answering over knowledge base

Shulin Cao, Jiaxin Shi, Liangming Pan, Lunyiu Nie, Yutong Xiang, Lei Hou, Juanzi Li, Bin He, and Hanwang Zhang. 2022 · 2022

Earlier work this paper cites.

Quality: Question answering with long input texts, yes!

Richard Yuanzhe Pang, Alicia Parrish, Nitish Joshi, Nikita Nangia, Jason Phang, Angelica Chen, Vishakh Padmakumar, Johnny Ma, Jana Thompson, He He, et al. 2022 · 2022

Earlier work this paper cites.

Squality: Building a long-document summarization dataset the hard way

Alex Wang, Richard Yuanzhe Pang, Angelica Chen, Jason Phang, and Samuel Bowman. 2022 · 2022

Earlier work this paper cites.

Chain-of-thought prompting elicits reasoning in large language models

Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. 2022 · 2022

Earlier work this paper cites.

Answering complex logical queries on knowledge graphs via query computation tree optimization

Yushi Bai, Xin Lv, Juanzi Li, and Lei Hou. 2023 · 2023

Earlier work this paper cites.

Mistral 7b

Original

Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al. 2023 · 2023

Earlier work this paper cites.

Needle in a haystack - pressure testing llms

Greg Kamradt. 2023 · 2023

Earlier work this paper cites.

Loogle: Can long-context language models understand long contexts?

Original

Jiaqi Li, Mengmeng Wang, Zilong Zheng, and Muhan Zhang. 2023 · 2023

Earlier work this paper cites.

Repobench: Benchmarking repository-level code auto-completion systems

Original

Tianyang Liu, Canwen Xu, and Julian McAuley. 2023 · 2023

Cited alongside, same era.

Gpqa: A graduate-level google-proof q&a benchmark

Original

David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R Bowman. 2023 · 2023

Cited alongside, same era.

Zeroscrolls: A zero-shot benchmark for long text understanding

Uri Shaham, Maor Ivgi, Avia Efrat, Jonathan Berant, and Omer Levy. 2023 · 2023

Cited alongside, same era.

Viskop: Visual knowledge oriented programming for interactive knowledge base question answering

Zijun Yao, Yuanyong Chen, Xin Lv, Shulin Cao, Amy Xin, Jifan Yu, Hailong Jin, Jianjun Xu, Peng Zhang, Lei Hou, et al. 2023 · 2023

Cited alongside, same era.

Judging llm-as-a-judge with mt-bench and chatbot arena

Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, et al. 2023 · 2023

Babilong: Testing the limits of llms with long context reasoning-in-a-haystack

Original

Yuri Kuratov, Aydar Bulatov, Petr Anokhin, Ivan Rodkin, Dmitry Sorokin, Artyom Sorokin, and Mikhail Burtsev. 2024 · 2024

Closest in time.

Summary of a haystack: A challenge to long-context llms and rag systems

Philippe Laban, Alexander Richard Fabbri, Caiming Xiong, and Chien-Sheng Wu. 2024 · 2024

Closest in time.

Long context rag performance of large language models

Original

Quinn Leng, Jacob Portes, Sam Havens, Matei Zaharia, and Michael Carbin. 2024 · 2024

Closest in time.

From generation to judgment: Opportunities and challenges of llm-as-a-judge

Original

Dawei Li, Bohan Jiang, Liangjie Huang, Alimohammad Beigi, Chengshuai Zhao, Zhen Tan, Amrita Bhattacharjee, Yuxuan Jiang, Canyu Chen, Tianhao Wu, et al. 2024 · 2024

Closest in time.

Longgenbench: Long-context generation benchmark