LIFBench: Evaluating the Instruction Following Performance and Stability of Large Language Models in Long-Context Scenarios

Some methods for classification and analysis of multivariate observations

J Macqueen. 1967 · 1967

Earlier work this paper cites.

Coefficient of variation

Charles E Brown. 1998 · 1998

Earlier work this paper cites.

Language models are few-shot learners

Original

Tom B Brown. 2020 · 2005

Earlier work this paper cites.

The coefficient of variation as an index of measurement reliability

Orit Shechtman. 2013 · 2013

Earlier work this paper cites.

Don’t give me the details, just the summary! topic-aware convolutional neural networks for extreme summarization

Original

Shashi Narayan, Shay B Cohen, and Mirella Lapata. 2018 · 2018

Earlier work this paper cites.

Efficient attentions for long document summarization

Original

Luyang Huang, Shuyang Cao, Nikolaus Parulian, Heng Ji, and Lu Wang. 2021 · 2021

Earlier work this paper cites.

What context features can transformer language models use?

Joe O’Connor and Jacob Andreas. 2021 · 2021

Earlier work this paper cites.

Qmsum: A new benchmark for query-based multi-domain meeting summarization

Original

Ming Zhong, Da Yin, Tao Yu, Ahmad Zaidi, Mutethia Mutuma, Rahul Jha, Ahmed Hassan Awadallah, Asli Celikyilmaz, Yang Liu, Xipeng Qiu, et al. 2021 · 2021

Earlier work this paper cites.

Scrolls: Standardized comparison over long language sequences

Uri Shaham, Elad Segal, Maor Ivgi, Avia Efrat, Ori Yoran, Adi Haviv, Ankit Gupta, Wenhan Xiong, Mor Geva, Jonathan Berant, et al. 2022 · 2022

Earlier work this paper cites.

Gpt-4 technical report

Original

Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023 · 2023

Earlier work this paper cites.

L-eval: Instituting standardized evaluation for long context language models

Original

Chenxin An, Shansan Gong, Ming Zhong, Xingjian Zhao, Mukai Li, Jun Zhang, Lingpeng Kong, and Xipeng Qiu. 2023 · 2023

Earlier work this paper cites.

Longbench: A bilingual, multitask benchmark for long context understanding

Original

Yushi Bai, Xin Lv, Jiajie Zhang, Hongchang Lyu, Jiankai Tang, Zhidian Huang, Zhengxiao Du, Xiao Liu, Aohan Zeng, Lei Hou, et al. 2023 · 2023

Earlier work this paper cites.

Palm: Scaling language modeling with pathways

Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, et al. 2023 · 2023

Earlier work this paper cites.

LLMTest: Needle in a Haystack

Greg Kamradt. 2023 · 2023

Earlier work this paper cites.

Unnatural instructions: Tuning language models with (almost) no human labor

Or Honovich, Thomas Scialom, Omer Levy, and Timo Schick. 2023 · 2023

Earlier work this paper cites.

Followbench: A multi-level fine-grained constraints following benchmark for large language models

Original

Yuxin Jiang, Yufei Wang, Xingshan Zeng, Wanjun Zhong, Liangyou Li, Fei Mi, Lifeng Shang, Xin Jiang, Qun Liu, and Wei Wang. 2023 · 2023

Earlier work this paper cites.

M4le: A multi-ability multi-range multi-task multi-domain long-context evaluation benchmark for large language models

Original

Wai-Chung Kwan, Xingshan Zeng, Yufei Wang, Yusen Sun, Liangyou Li, Lifeng Shang, Qun Liu, and Kam-Fai Wong. 2023 · 2023

Cited alongside, same era.

Efficient memory management for large language model serving with pagedattention

Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph Gonzalez, Hao Zhang, and Ion Stoica. 2023 · 2023

Cited alongside, same era.

Loogle: Can long-context language models understand long contexts?

Original

Jiaqi Li, Mengmeng Wang, Zilong Zheng, and Muhan Zhang. 2023 · 2023

Cited alongside, same era.

Yarn: Efficient context window extension of large language models

Original

Bowen Peng, Jeffrey Quesnelle, Honglu Fan, and Enrico Shippole. 2023 · 2023

Cited alongside, same era.

Zeroscrolls: A zero-shot benchmark for long text understanding

Can large language models understand real-world complex instructions?

Qianyu He, Jie Zeng, Wenhao Huang, Lina Chen, Jin Xiao, Qianxi He, Xunzhe Zhou, Jiaqing Liang, and Yanghua Xiao. 2024 · 2024

Closest in time.

Ruler: What’s the real context size of your long-context language models?

Original

Cheng-Ping Hsieh, Simeng Sun, Samuel Kriman, Shantanu Acharya, Dima Rekesh, Fei Jia, and Boris Ginsburg. 2024 · 2024

Closest in time.

Same task, more tokens: the impact of input length on the reasoning performance of large language models

Original

Mosh Levy, Alon Jacoby, and Yoav Goldberg. 2024 · 2024

Closest in time.

Mathvista: Evaluating mathematical reasoning of foundation models in visual contexts

Pan Lu, Hritik Bansal, Tony Xia, Jiacheng Liu, Chunyuan Li, Hannaneh Hajishirzi, Hao Cheng, Kai-Wei Chang, Michel Galley, and Jianfeng Gao. 2024 · 2024

Closest in time.

Xl 2 bench: A benchmark for extremely long context understanding with long-range dependencies