Fetching the paper…
Reading the bibliography…
While the language modeling objective has been shown to be deeply connected with compression, it is surprising that modern LLMs are not employed in practical text compression systems.
Data compression using adaptive coding and partial string matching
J. Cleary and I. Witten. 1984 · 1984
Earlier work this paper cites.
Sequential neural text compression
J. Schmidhuber and S. Heil. 1996 · 1996
Earlier work this paper cites.
Fast text compression with neural networks
Matthew V. Mahoney. 2000 · 2000
Earlier work this paper cites.
Deepzip: Lossless data compression using recurrent neural networks
Mohit Goyal, Kedar Tatwawadi, Shubham Chandak, and Idoia Ochoa. 2018 · 2018
Earlier work this paper cites.
Lossless data compression with neural networks
Fabrice Bellard. 2019 · 2019
Earlier work this paper cites.
Language models are unsupervised multitask learners
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019 · 2019
Earlier work this paper cites.
Nncp v2: Lossless data compression with transformer
Fabrice Bellard. 2021 · 2021
Cited alongside, same era.
Lora: Low-rank adaptation of large language models
Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2021 · 2021
Cited alongside, same era.
Peft: State-of-the-art parameter-efficient fine-tuning methods
Sourab Mangrulkar, Sylvain Gugger, Lysandre Debut, Younes Belkada, Sayak Paul, and Benjamin Bossan. 2022 · 2022
Cited alongside, same era.
Qlora: Efficient finetuning of quantized llms
Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. 2023 · 2023
Cited alongside, same era.
Llama 2: Open foundation and fine-tuned chat models, 2023
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. 2023b · 2023
Language modeling is compression
Grégoire Delétang, Anian Ruoss, Paul-Ambroise Duquenne, Elliot Catt, Tim Genewein, Christopher Mattern, Jordi Grau-Moya, Li Kevin Wenliang, Matthew Aitchison, Laurent Orseau, Marcus Hutter, and Joel Veness. 2024 · 2024
Closest in time.
Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. 2024 · 2024
Closest in time.
Brotli compression algorithm
Google. 2024 · 2024
Closest in time.
Compression represents intelligence linearly
Yuzhen Huang, Jinghan Zhang, Zifei Shan, and Junxian He. 2024 · 2024
Closest in time.
Zlib: A massively spiffy yet delicately unobtrusive compression library
Jean-loup Gailly. 2024 · 2024
Closest in time.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Cited alongside, same era.
Llmzip: Lossless text compression using large language models
Chandra Shekhara Kaushik Valmeekam, Krishna Narayanan, Dileep Kalathil, Jean-Francois Chamberland, and Srinivas Shakkottai. 2023 · 2023
Cited alongside, same era.
Llama: Open and efficient foundation language models
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. 2023a
Cited in the paper.
bzip2 - a free and open-source file compression program
Julian Seward. 2024 · 2024
Closest in time.