Fetching the paper…
Reading the bibliography…
Generative inference in Large Language Models (LLMs) is impeded by the growing memory demands of Key-Value (KV) cache, especially for longer sequences.
Cache memories
Alan Jay Smith · 1982
Earlier work this paper cites.
The exponentially weighted moving average
J. Stuart Hunter · 1986
Earlier work this paper cites.
Large scale many-objective optimization driven by distributional adversarial networks
Zhenyu Liang, Yunfan Li, and Zhongwei Wan · 2003
Earlier work this paper cites.
Many-objective estimation of distribution optimization algorithm based on wgan-gp
Zhenyu Liang, Yunfan Li, and Zhongwei Wan · 2003
Earlier work this paper cites.
Elements of information theory
MTCAJ Thomas and A Thomas Joy · 2006
Earlier work this paper cites.
Information theory
Robert B Ash · 2012
Earlier work this paper cites.
Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio · 2017
Earlier work this paper cites.
fairseq: A fast, extensible toolkit for sequence modeling
Myle Ott, Sergey Edunov, Alexei Baevski, Angela Fan, Sam Gross, Nathan Ng, David Grangier, and Michael Auli · 2019
Earlier work this paper cites.
Compressive transformers for long-range sequence modelling
Jack W Rae, Anna Potapenko, Siddhant M Jayakumar, and Timothy P Lillicrap · 2019
Earlier work this paper cites.
Coqa: A conversational question answering challenge
Siva Reddy, Danqi Chen, and Christopher D. Manning · 2019
Earlier work this paper cites.
Fast transformer decoding: One write-head is all you need
Noam Shazeer · 2019
Earlier work this paper cites.
Huggingface’s transformers: State-of-the-art natural language processing
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, et al · 2019
Earlier work this paper cites.
Transformers: State-of-the-art natural language processing
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, et al · 2020
Earlier work this paper cites.
Nearest neighbor matching for deep clustering
Zhiyuan Dang, Cheng Deng, Xu Yang, Kun Wei, and Heng Huang · 2021
Earlier work this paper cites.
A framework for few-shot language model evaluation
Leo Gao, Jonathan Tow, Stella Biderman, Sid Black, Anthony DiPofi, Charles Foster, Laurence Golding, Jeffrey Hsu, Kyle McDonell, Niklas Muennighoff, et al · 2021
Earlier work this paper cites.
Token merging: Your vit but faster
Daniel Bolya, Cheng-Yang Fu, Xiaoliang Dai, Peizhao Zhang, Christoph Feichtenhofer, and Judy Hoffman · 2022
Earlier work this paper cites.
Not all patches are what you need: Expediting vision transformers via token reorganizations
Youwei Liang, Chongjian Ge, Zhan Tong, Yibing Song, Jue Wang, and Pengtao Xie · 2022
Earlier work this paper cites.
Truthfulqa: Measuring how models mimic human falsehoods
Stephanie Lin, Jacob Hilton, and Owain Evans · 2022
Earlier work this paper cites.
Compression of generative pre-trained language models via quantization
Chaofan Tao, Lu Hou, Wei Zhang, Lifeng Shang, Xin Jiang, Qun Liu, Ping Luo, and Ngai Wong · 2022
Earlier work this paper cites.
G-map: general memory-augmented pre-trained language model for domain tasks
Zhongwei Wan, Yichun Yin, Wei Zhang, Jiaxin Shi, Lifeng Shang, Guangyong Chen, Xin Jiang, and Qun Liu · 2022
Earlier work this paper cites.
Self-consistent reasoning for solving math word problems
Jing Xiong, Zhongwei Wan, Xiping Hu, Min Yang, and Chengming Li · 2022
Cited alongside, same era.
Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al · 2023
Cited alongside, same era.
Gqa: Training generalized multi-query transformer models from multi-head checkpoints
Joshua Ainslie, James Lee-Thorp, Michiel de Jong, Yury Zemlyanskiy, Federico Lebrón, and Sumit Sanghai · 2023
Cited alongside, same era.
The falcon series of open language models
Ebtesam Almazrouei, Hamza Alobeidli, Abdulaziz Alshamsi, Alessandro Cappelli, Ruxandra Cojocaru, Mérouane Debbah, Étienne Goffinet, Daniel Hesslow, Julien Launay, Quentin Malartic, et al · 2023
Cited alongside, same era.
Get more with less: Synthesizing recurrence with kv cache compression for efficient llm inference
Harry Dong, Xinyu Yang, Zhenyu Zhang, Zhangyang Wang, Yuejie Chi, and Beidi Chen · 2024
Closest in time.
Neuroclips: Towards high-fidelity and smooth fmri-to-video reconstruction
Zixuan Gong, Guangyin Bao, Qi Zhang, Zhongwei Wan, Duoqian Miao, Shoujin Wang, Lei Zhu, Changwei Wang, Rongtao Xu, Liang Hu, et al · 2024
Closest in time.
Ruler: What’s the real context size of your long-context language models?
Cheng-Ping Hsieh, Simeng Sun, Samuel Kriman, Shantanu Acharya, Dima Rekesh, Fei Jia, Yang Zhang, and Boris Ginsburg · 2024
Closest in time.
Evolver: Chain-of-evolution prompting to boost large multimodal models for hateful meme detection
Jinfa Huang, Jinsheng Pan, Zhongwei Wan, Hanjia Lyu, and Jiebo Luo · 2024
Closest in time.
Etp: Learning transferable ecg representations via ecg-text pre-training
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Yushi Bai, Xin Lv, Jiajie Zhang, Hongchang Lyu, Jiankai Tang, Zhidian Huang, Zhengxiao Du, Xiao Liu, Aohan Zeng, Lei Hou, et al · 2023
Cited alongside, same era.
Eleftheria Briakou, Colin Cherry, and George Foster · 2023
Cited alongside, same era.
How to scale your EMA
Dan Busbridge, Jason Ramapuram, Pierre Ablin, Tatiana Likhomanenko, Eeshan Gunesh Dhekane, Xavier Suau Cuadros, and Russell Webb · 2023
Cited alongside, same era.
Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality, march 2023
Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E Gonzalez, et al · 2023
Cited alongside, same era.
Model tells you what to discard: Adaptive kv cache compression for llms
Suyu Ge, Yunan Zhang, Liyuan Liu, Minjia Zhang, Jiawei Han, and Jianfeng Gao · 2023
Cited alongside, same era.
Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al · 2023
Cited alongside, same era.
Evaluating open-domain question answering in the era of large language models
Ehsan Kamalloo, Nouha Dziri, Charles LA Clarke, and Davood Rafiei · 2023
Cited alongside, same era.
Needle in a haystack–pressure testing llms, 2023
G Kamradt · 2023
Cited alongside, same era.
Che Liu, Zhongwei Wan, Sibo Cheng, Mi Zhang, and Rossella Arcucci · 2024
Closest in time.
Contemporary model compression on large language models inference
Dong Liu · 2024
Closest in time.
Introducing meta llama 3: The most capable openly available llm to date, 2024
Meta · 2024
Closest in time.
Dynamic memory compression: Retrofitting llms for accelerated inference
Piotr Nawrot, Adrian Łańcucki, Marcin Chochowski, David Tarjan, and Edoardo M Ponti · 2024
Closest in time.
On the efficacy of eviction policy for key-value constrained generative language model inference
Siyu Ren and Kenny Q. Zhu · 2024
Closest in time.
Famba-v: Fast vision mamba with cross-layer token fusion
Hui Shen, Zhongwei Wan, Xin Wang, and Mi Zhang · 2024
Closest in time.
Scaling laws with vocabulary: Larger models deserve larger vocabularies
Chaofan Tao, Qian Liu, Longxu Dou, Niklas Muennighoff, Zhongwei Wan, Ping Luo, Min Lin, and Ngai Wong · 2024
Closest in time.
V-petl bench: A unified visual parameter-efficient transfer learning benchmark
Yi Xin, Siqi Luo, Xuyang Liu, Haodi Zhou, Xinyu Cheng, Christina E Lee, Junlong Du, Haozhe Wang, MingCai Chen, Ting Liu, et al · 2024
Closest in time.
Sirllm: Streaming infinite retentive llm
Yao Yao, Zuchao Li, and Hai Zhao · 2024
Closest in time.
Dglf: A dual graph-based learning framework for multi-modal sarcasm detection
Zhihong Zhu, Kefan Shen, Zhaorun Chen, Yunyan Zhang, Yuyan Chen, Xiaoqi Jiao, Zhongwei Wan, Shaorong Xie, Wei Liu, Xian Wu, et al · 2024
Closest in time.
Simin Chen, Yiming Chen, Zexin Li, Yifan Jiang, Zhongwei Wan, Yixin He, Dezhi Ran, Tianle Gu, Haizhou Li, Tao Xie, et al · 2025
Closest in time.
Knowledge-enhanced multimodal ecg representation learning with arbitrary-lead inputs
Che Liu, Cheng Ouyang, Zhongwei Wan, Haozhe Wang, Wenjia Bai, and Rossella Arcucci · 2025
Closest in time.
Efficient diffusion models: A survey
Hui Shen, Jingxuan Zhang, Boning Xiong, Rui Hu, Shoufa Chen, Zhongwei Wan, Xin Wang, Yu Zhang, Zixuan Gong, Guangyin Bao, et al · 2025
Closest in time.
Meda: Dynamic kv cache allocation for efficient multimodal long-context inference
Zhongwei Wan, Hui Shen, Xin Wang, Che Liu, Zheda Mai, and Mi Zhang · 2025
Closest in time.
Parallelcomp: Parallel long-context compressor for length extrapolation
Jing Xiong, Jianghan Shen, Chuanyang Zheng, Zhongwei Wan, Chenyang Zhao, Chiwun Yang, Fanghua Ye, Hongxia Yang, Lingpeng Kong, and Ngai Wong · 2025
Closest in time.