Fetching the paper…
Reading the bibliography…
As Large Language Models (LLMs) continue to evolve, more are being designed to handle long-context inputs.
Measuring Massive Multitask Language Understanding
Hendrycks, D.; Burns, C.; Basart, S.; Zou, A.; Mazeika, M.; Song, D.; and Steinhardt, J. 2021 · 2009
Earlier work this paper cites.
Introduction to Common Crawl Datasets
Patel, J. M. 2020 · 2020
Earlier work this paper cites.
RoFormer: Enhanced Transformer with Rotary Position Embedding
Su, J.; Lu, Y.; Pan, S.; Murtadha, A.; Wen, B.; and Liu, Y. 2022 · 2022
Earlier work this paper cites.
L-Eval: Instituting Standardized Evaluation for Long Context Language Models
An, C.; Gong, S.; Zhong, M.; Zhao, X.; Li, M.; Zhang, J.; Kong, L.; and Qiu, X. 2023 · 2023
Earlier work this paper cites.
Long context prompting for Claude 2.1
Anthropic. 2023 · 2023
Earlier work this paper cites.
Wudao Corpus
BAAI. 2023 · 2023
Earlier work this paper cites.
QLoRA: Efficient Finetuning of Quantized LLMs
Dettmers, T.; Pagnoni, A.; Holtzman, A.; and Zettlemoyer, L. 2023 · 2023
Earlier work this paper cites.
FlagAlpha/Llama2-Chinese
FlagAlpha. 2023 · 2023
Earlier work this paper cites.
Empower Your Model with Longer and Better Context Comprehension
Gao, Y.; Wang, L.; Fang, J.; Hu, L.; and Cheng, J. 2023 · 2023
Earlier work this paper cites.
LLMTest_NeedleInAHaystack: Doing simple retrieval from LLM models at various context lengths to measure accuracy
gkamradt. 2023 · 2023
Cited alongside, same era.
He, J.; Pan, K.; Dong, X.; Song, Z.; Liu, Y.; Liang, Y.; Wang, H.; Sun, Q.; Zhang, S.; Xie, Z.; and Zhang, J. 2023 · 2023
Cited alongside, same era.
DachengLi1/LongChat
Li, D. 2024 · 2023
Cited alongside, same era.
LooGLE: Can Long-Context Language Models Understand Long Contexts?
Li, J.; Wang, M.; Zheng, Z.; and Zhang, M. 2023 · 2023
Cited alongside, same era.
Lost in the Middle: How Language Models Use Long Contexts
Liu, N. F.; Lin, K.; Hewitt, J.; Paranjape, A.; Bevilacqua, M.; Petroni, F.; and Liang, P. 2023 · 2023
togethercomputer/Long-Data-Collections · Datasets at Hugging Face
Together. 2023 · 2023
Closest in time.
Llama 2: Open Foundation and Fine-Tuned Chat Models
Touvron, H.; Martin, L.; Stone, K.; Albert, P.; Almahairi, A.; Babaei, Y.; Bashlykov, N.; Batra, S.; Bhargava, P.; Bhosale, S.; Bikel, D.; Blecher, L.; Ferrer, C. C.; Chen, M.; Cucurull, G.; Esiobu, D.; Fernandes, J.; Fu, J.; Fu, W.; Fuller, B.; Gao, C.; Goswami, V.; Goyal, N.; Hartshorn, A.; Hosseini, S.; Hou, R.; Inan, H.; Kardas, M.; Kerkez, V.; Khabsa, M.; Kloumann, I.; Korenev, A.; Koura, P. S.; Lachaux, M.-A.; Lavril, T.; Lee, J.; Liskovich, D.; Lu, Y.; Mao, Y.; Martinet, X.; Mihaylov, T.; Mishra, P.; Molybog, I.; Nie, Y.; Poulton, A.; Reizenstein, J.; Rungta, R.; Saladi, K.; Schelten, A.; Silva, R.; Smith, E. M.; Subramanian, R.; Tan, X. E.; Tang, B.; Taylor, R.; Williams, A.; Kuan, J. X.; Xu, P.; Yan, Z.; Zarov, I.; Zhang, Y.; Fan, A.; Kambadur, M.; Narang, S.; Rodriguez, A.; Stojnic, R.; Edunov, S.; and Scialom, T. 2023 · 2023
Closest in time.
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
Wei, J.; Wang, X.; Schuurmans, D.; Bosma, M.; Ichter, B.; Xia, F.; Chi, E.; Le, Q.; and Zhou, D. 2023 · 2023
Closest in time.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Cited alongside, same era.
OpenAI. 2023 · 2023
Cited alongside, same era.
YaRN: Efficient Context Window Extension of Large Language Models
Peng, B.; Quesnelle, J.; Fan, H.; and Shippole, E. 2023 · 2023
Cited alongside, same era.
Attention Sorting Combats Recency Bias In Long Context Language Models
Peysakhovich, A.; and Lerer, A. 2023 · 2023
Cited alongside, same era.
Stanford Alpaca: An Instruction-following LLaMA model
Rohan Taori; Ishaan Gulrajani; and Tianyi Zhang. 2023 · 2023
Cited alongside, same era.
Bai, J.; Bai, S.; Chu, Y.; Cui, Z.; Dang, K.; Deng, X.; Fan, Y.; Ge, W.; Han, Y.; Huang, F.; Hui, B.; Ji, L.; Li, M.; Lin, J.; Lin, R.; Liu, D.; Liu, G.; Lu, C.; Lu, K.; Ma, J.; Men, R.; Ren, X.; Ren, X.; Tan, C.; Tan, S.; Tu, J.; Wang, P.; Wang, S.; Wang, W.; Wu, S.; Xu, B.; Xu, J.; Yang, A.; Yang, H.; Yang, J.; Yang, S.; Yao, Y.; Yu, B.; Yuan, H.; Yuan, Z.; Zhang, J.; Zhang, X.; Zhang, Y.; Zhang, Z.; Zhou, C.; Zhou, J.; Zhou, X.; and Zhu, T. 2023a
Cited in the paper.
LongBench: A Bilingual, Multitask Benchmark for Long Context Understanding
Bai, Y.; Lv, X.; Zhang, J.; Lyu, H.; Tang, J.; Huang, Z.; Du, Z.; Liu, X.; Zeng, A.; Hou, L.; Dong, Y.; Tang, J.; and Li, J. 2023b
Cited in the paper.
Extending Context Window of Large Language Models via Positional Interpolation
Chen, S.; Wong, S.; Chen, L.; and Tian, Y. 2023a
Cited in the paper.
An, S.; Ma, Z.; Lin, Z.; Zheng, N.; and Lou, J.-G. 2024 · 2024
Closest in time.
Found in the Middle: Calibrating Positional Attention Bias Improves Long Context Utilization
Hsieh, C.-Y.; Chuang, Y.-S.; Li, C.-L.; Wang, Z.; Le, L. T.; Kumar, A.; Glass, J.; Ratner, A.; Lee, C.-Y.; Krishna, R.; and Pfister, T. 2024 · 2024
Closest in time.
Leave No Document Behind: Benchmarking Long-Context LLMs with Extended Multi-Doc QA
Wang, M.; Chen, L.; Fu, C.; Liao, S.; Zhang, X.; Wu, B.; Yu, H.; Xu, N.; Zhang, L.; Luo, R.; Li, Y.; Yang, M.; Huang, F.; and Li, Y. 2024 · 2024
Closest in time.
Yang, A.; Yang, B.; Hui, B.; Zheng, B.; Yu, B.; Zhou, C.; Li, C.; Li, C.; Liu, D.; Huang, F.; Dong, G.; Wei, H.; Lin, H.; Tang, J.; Wang, J.; Yang, J.; Tu, J.; Zhang, J.; Ma, J.; Yang, J.; Xu, J.; Zhou, J.; Bai, J.; He, J.; Lin, J.; Dang, K.; Lu, K.; Chen, K.; Yang, K.; Li, M.; Xue, M.; Ni, N.; Zhang, P.; Wang, P.; Peng, R.; Men, R.; Gao, R.; Lin, R.; Wang, S.; Bai, S.; Tan, S.; Zhu, T.; Li, T.; Liu, T.; Ge, W.; Deng, X.; Zhou, X.; Ren, X.; Zhang, X.; Wei, X.; Ren, X.; Liu, X.; Fan, Y.; Yao, Y.; Zhang, Y.; Wan, Y.; Chu, Y.; Liu, Y.; Cui, Z.; Zhang, Z.; Guo, Z.; and Fan, Z. 2024 · 2024
Closest in time.
Zhang, Z.; Chen, R.; Liu, S.; Yao, Z.; Ruwase, O.; Chen, B.; Wu, X.; and Wang, Z. 2024 · 2024
Closest in time.