InfiniPot: Infinite Context Processing on Memory-Constrained LLMs

Fast transformer decoding: One write-head is all you need

Original

Noam Shazeer. 2019 · 1911

Earlier work this paper cites.

Longformer: The long-document transformer

Original

Iz Beltagy, Matthew E. Peters, and Arman Cohan. 2020 · 2004

Earlier work this paper cites.

Block-recurrent transformers

DeLesley Hutchins, Imanol Schlag, Yuhuai Wu, Ethan Dyer, and Behnam Neyshabur. 2022 · 2022

Earlier work this paper cites.

Gqa: Training generalized multi-query transformer models from multi-head checkpoints

Original

Joshua Ainslie, James Lee-Thorp, Michiel de Jong, Yury Zemlyanskiy, Federico Lebrón, and Sumit Sanghai. 2023 · 2023

Earlier work this paper cites.

Longbench: A bilingual, multitask benchmark for long context understanding

Original

Yushi Bai, Xin Lv, Jiajie Zhang, Hongchang Lyu, Jiankai Tang, Zhidian Huang, Zhengxiao Du, Xiao Liu, Aohan Zeng, Lei Hou, Yuxiao Dong, Jie Tang, and Juanzi Li. 2023 · 2023

Earlier work this paper cites.

Extending context window of large language models via positional interpolation

Original

Shouyuan Chen, Sherman Wong, Liangjian Chen, and Yuandong Tian. 2023 · 2023

Earlier work this paper cites.

Lm-infinite: Simple on-the-fly length generalization for large language models

Original

Chi Han, Qifan Wang, Wenhan Xiong, Yu Chen, Heng Ji, and Sinong Wang. 2023 · 2023

Earlier work this paper cites.

Advancing transformer architecture in long-context large language models: A comprehensive survey

Original

Yunpeng Huang, Jingwei Xu, Zixu Jiang, Junyu Lai, Zenan Li, Yuan Yao, Taolue Chen, Lijuan Yang, Zhou Xin, and Xiaoxing Ma. 2023 · 2023

Earlier work this paper cites.

Mistral 7b

Original

Albert Qiaochu Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de Las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, L’elio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timothée Lacroix, and William El Sayed. 2023 · 2023

Earlier work this paper cites.

Needle in a haystack–pressure testing llms

G Kamradt. 2023 · 2023

Earlier work this paper cites.

Full stack optimization of transformer inference

Sehoon Kim, Coleman Hooper, Thanakul Wattanawong, Minwoo Kang, Ruohan Yan, Hasan Genc, Grace Dinh, Qijing Huang, Kurt Keutzer, Michael W. Mahoney, Sophia Shao, and Amir Gholami. 2023 · 2023

Earlier work this paper cites.

Loogle: Can long-context language models understand long contexts?

Original

Jiaqi Li, Mengmeng Wang, Zilong Zheng, and Muhan Zhang. 2023 · 2023

Earlier work this paper cites.

Estimating the carbon footprint of bloom, a 176b parameter language model

Alexandra Sasha Luccioni, Sylvain Viguier, and Anne-Laure Ligozat. 2023 · 2023

Earlier work this paper cites.

Landmark attention: Random-access infinite context length for transformers

Amirkeivan Mohtashami and Martin Jaggi. 2023 · 2023

Earlier work this paper cites.

When does in-context learning fall short and why? a study on specification-heavy tasks

Original

Hao Peng, Xiaozhi Wang, Jianhui Chen, Weikai Li, Yunjia Qi, Zimu Wang, Zhili Wu, Kaisheng Zeng, Bin Xu, Lei Hou, et al. 2023 · 2023

Earlier work this paper cites.

Efficiently scaling transformer inference

Reiner Pope, Sholto Douglas, Aakanksha Chowdhery, Jacob Devlin, James Bradbury, Jonathan Heek, Kefan Xiao, Shivani Agrawal, and Jeff Dean. 2023 · 2023

Cited alongside, same era.

H2o: Heavy-hitter oracle for efficient generative inference of large language models

Zhenyu Zhang, Ying Sheng, Tianyi Zhou, Tianlong Chen, Lianmin Zheng, Ruisi Cai, Zhao Song, Yuandong Tian, Christopher Re, Clark Barrett, Zhangyang Wang, and Beidi Chen. 2023 · 2023

Cited alongside, same era.

Phi-3 technical report: A highly capable language model locally on your phone

Marah Abdin and et al. 2024 · 2024

Cited alongside, same era.

Introducing the next generation of claude

Anthropic. 2024 · 2024

Cited alongside, same era.

Demystifying platform requirements for diverse llm inference use cases

Original

Abhimanyu Bambhaniya, Ritik Raj, Geonhwa Jeong, Souvik Kundu, Sudarshan Srinivasan, Midhilesh Elavazhagan, Madhu Kumar, and Tushar Krishna. 2024 · 2024

Cited alongside, same era.

Kvquant: Towards 10 million context length llm inference with kv cache quantization

Original

Coleman Hooper, Sehoon Kim, Hiva Mohammadzadeh, Michael W. Mahoney, Yakun Sophia Shao, Kurt Keutzer, and Amir Gholami. 2024 · 2024

Closest in time.

Transformerfam: Feedback attention is working memory

Original

Dongseong Hwang, Weiran Wang, Zhuoyuan Huo, Khe Chai Sim, and Pedro Moreno Mengibar. 2024 · 2024

Closest in time.

Llm maybe longlm: Self-extend llm context window without tuning

Original

Hongye Jin, Xiaotian Han, Jingfeng Yang, Zhimeng Jiang, Zirui Liu, Chia-Yuan Chang, Huiyuan Chen, and Xia Hu. 2024 · 2024

Closest in time.

The impact of positional encoding on length generalization in transformers

Amirhossein Kazemnejad, Inkit Padhi, Karthikeyan Natesan Ramamurthy, Payel Das, and Siva Reddy. 2024 · 2024

Closest in time.

Snapkv: Llm knows what you are looking for before generation