Built on

Shannon, C.E.: A mathematical theory of communication. The Bell System Technical Journal 27
1948
Earlier work this paper cites.
Solomonoff, R.: A formal theory of inductive inference. Inform. control 7
1964
Earlier work this paper cites.
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al
2015
Earlier work this paper cites.
Panayotov, V., Chen, G., Povey, D., Khudanpur, S.: Librispeech: An asr corpus based on public domain audio book. In: 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (2015)
2015
Earlier work this paper cites.
Ballé, J., Laparra, V., Simoncelli, E.P.: End-to-end optimized image compression. arXiv preprint arXiv:1611.01704 (2016)
Original
2016
Earlier work this paper cites.
Ito, K., Johnson, L.: The LJ Speech Dataset (2017). https://keithito.com/LJ-Speech-Dataset/
2017
Earlier work this paper cites.
Santurkar, S., Budden, D., Shavit, N.: Generative compression. In: 2018 Picture Coding Symposium (PCS), pp. 258–262 (2018). IEEE
2018
Earlier work this paper cites.
CLIC: Workshop and challenge on learned image compression (2019). https://clic.compression.cc/2019/
2019
Earlier work this paper cites.

Similar

Chen, M., Radford, A., Child, R., Wu, J., Jun, H., Luan, D., Sutskever, I.: Generative pretraining from pixels. In: International Conference on Machine Learning, pp. 1691–1703 (2020). PMLR
2020
Cited alongside, same era.
Wen, Z., Lu, X.H., Reddy, S.: Medal: Medical abbreviation disambiguation dataset for natural language understanding pretraining. arXiv preprint arXiv:2012.13978 (2020)
Original
2020
Cited alongside, same era.
Bellard, F.: NNCP v2: Lossless data compression with transformer (2021). https://bellard.org/nncp/nncp_v2.pdf
2021
Cited alongside, same era.
Li, J., Li, B., Lu, Y.: Deep contextual video compression. Advances in Neural Information Processing Systems 34
2021
Cited alongside, same era.
Chung, H., Kim, J., Mccann, M.T., Klasky, M.L., Ye, J.C.: Diffusion posterior sampling for general noisy inverse problems. arXiv preprint arXiv:2209.14687 (2022)

Then

Henderson, P., Krass, M., Zheng, L., Guha, N., Manning, C., Jurafsky, D., Ho, D.E.: Pile of law: Learning responsible data filtering from the law and a 256gb open-source legal dataset (2022) https://doi.org/10.48550/arXiv.2207.00220
2022
Later among the works it cites.
Huang, C., Xie, Y., Jiang, Z., Lin, J., Li, M.: Approximating human-like few-shot learning with gpt-based compression. arXiv preprint arXiv:2308.06942 (2023)
Original
2023
Later among the works it cites.
Delétang, G., Ruoss, A., Duquenne, P.-A., Catt, E., Genewein, T., Mattern, C., Grau-Moya, J., Wenliang, L.K., Aitchison, M., Orseau, L., et al.: Language modeling is compression. arXiv preprint arXiv:2309.10668 (2023)
Original
2023
Later among the works it cites.
Touvron, H., Lavril, T., Izacard, G., Lachaux, X.M.M.-A., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., Azhar, F., Rodriguez, A., Joulin, A., Grave, E., Lample, G.: LLaMA: Open and Efficient Foundation Language Models (2023). https://research.facebook.com/file/1574548786327032/LLaMA--Open-and-Efficient-Foundation-Language-Models.pdf

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…

2024

Lossless data compression by large models

Li, Ziguang, Huang, Chao, Wang, Xuliang et al.

Understand

Modern data compression methods are slowly reaching their limits after 80 years of research, millions of papers, and wide range of applications.

Yet, the extravagant 6G communication speed requirement raises a major open question for revolutionary new ideas of data compression.
We have previously shown all understanding or learning are compression, under reasonable assumptions.
Large language models (LLMs) understand data better than ever before.

Built on

Shannon, C.E.: A mathematical theory of communication. The Bell System Technical Journal 27
1948
Earlier work this paper cites.
Solomonoff, R.: A formal theory of inductive inference. Inform. control 7
1964
Earlier work this paper cites.
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al
2015
Earlier work this paper cites.
Panayotov, V., Chen, G., Povey, D., Khudanpur, S.: Librispeech: An asr corpus based on public domain audio book. In: 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (2015)
2015
Earlier work this paper cites.
Ballé, J., Laparra, V., Simoncelli, E.P.: End-to-end optimized image compression. arXiv preprint arXiv:1611.01704 (2016)
Original
2016
Earlier work this paper cites.
Ito, K., Johnson, L.: The LJ Speech Dataset (2017). https://keithito.com/LJ-Speech-Dataset/
2017
Earlier work this paper cites.
Santurkar, S., Budden, D., Shavit, N.: Generative compression. In: 2018 Picture Coding Symposium (PCS), pp. 258–262 (2018). IEEE
2018
Earlier work this paper cites.
CLIC: Workshop and challenge on learned image compression (2019). https://clic.compression.cc/2019/
2019
Earlier work this paper cites.

Similar

Chen, M., Radford, A., Child, R., Wu, J., Jun, H., Luan, D., Sutskever, I.: Generative pretraining from pixels. In: International Conference on Machine Learning, pp. 1691–1703 (2020). PMLR
2020
Cited alongside, same era.
Wen, Z., Lu, X.H., Reddy, S.: Medal: Medical abbreviation disambiguation dataset for natural language understanding pretraining. arXiv preprint arXiv:2012.13978 (2020)
Original
2020
Cited alongside, same era.
Bellard, F.: NNCP v2: Lossless data compression with transformer (2021). https://bellard.org/nncp/nncp_v2.pdf
2021
Cited alongside, same era.
Li, J., Li, B., Lu, Y.: Deep contextual video compression. Advances in Neural Information Processing Systems 34
2021
Cited alongside, same era.
Chung, H., Kim, J., Mccann, M.T., Klasky, M.L., Ye, J.C.: Diffusion posterior sampling for general noisy inverse problems. arXiv preprint arXiv:2209.14687 (2022)

Then

Henderson, P., Krass, M., Zheng, L., Guha, N., Manning, C., Jurafsky, D., Ho, D.E.: Pile of law: Learning responsible data filtering from the law and a 256gb open-source legal dataset (2022) https://doi.org/10.48550/arXiv.2207.00220
2022
Later among the works it cites.
Huang, C., Xie, Y., Jiang, Z., Lin, J., Li, M.: Approximating human-like few-shot learning with gpt-based compression. arXiv preprint arXiv:2308.06942 (2023)
Original
2023
Later among the works it cites.
Delétang, G., Ruoss, A., Duquenne, P.-A., Catt, E., Genewein, T., Mattern, C., Grau-Moya, J., Wenliang, L.K., Aitchison, M., Orseau, L., et al.: Language modeling is compression. arXiv preprint arXiv:2309.10668 (2023)
Original
2023
Later among the works it cites.
Touvron, H., Lavril, T., Izacard, G., Lachaux, X.M.M.-A., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., Azhar, F., Rodriguez, A., Joulin, A., Grave, E., Lample, G.: LLaMA: Open and Efficient Foundation Language Models (2023). https://research.facebook.com/file/1574548786327032/LLaMA--Open-and-Efficient-Foundation-Language-Models.pdf

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Original

2022

Cited alongside, same era.

Rhee, H., Jang, Y.I., Kim, S., Cho, N.I.: Lc-fdnet: Learned lossless image compression with frequency decomposition network. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6033–6042 (2022)

2022

Cited alongside, same era.

He, D., Yang, Z., Peng, W., Ma, R., Qin, H., Wang, Y.: Elic: Efficient learned image compression with unevenly grouped space-channel contextual adaptive coding. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5718–5727 (2022)

2022

Cited alongside, same era.

Hu, E.J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., Chen, W.: Lora: Low-rank adaptation of large language models. In: ICLR 2022 (2022)

2022

Cited alongside, same era.

2023

Later among the works it cites.

Jiang, Z., Wang, R., Bu, D., Li, M.: A theory of human-like few-shot learning. arXiv preprint arXiv:2301.01047 (2023)

Original

2023

Later among the works it cites.

Yang, R., Mandt, S.: Lossy image compression with conditional diffusion models. Advances in Neural Information Processing Systems 36

2024

Closest in time.

Relic, L., Azevedo, R., Gross, M., Schroers, C.: Lossy image compression with foundation diffusion models. arXiv preprint arXiv:2404.08580 (2024)

Original

2024

Closest in time.

Xu, T., Zhu, Z., He, D., Li, Y., Guo, L., Wang, Y., Wang, Z., Qin, H., Wang, Y., Liu, J., et al.: Idempotence and perceptual image compression. arXiv preprint arXiv:2401.08920 (2024)

Original

2024

Closest in time.