Fetching the paper…
Reading the bibliography…
Large language models (LLMs) have triggered a new stream of research focusing on compressing the context length to reduce the computational cost while ensuring the retention of helpful information for LLMs to answer the given question.
Longformer: The long-document transformer
Beltagy, I.; Peters, M. E.; and Cohan, A. 2020 · 2004
Earlier work this paper cites.
Rouge: A package for automatic evaluation of summaries
Lin, C.-Y. 2004 · 2004
Earlier work this paper cites.
A normalized Levenshtein distance metric
Yujian, L.; and Bo, L. 2007 · 2007
Earlier work this paper cites.
Large dataset for keyphrases extraction
Krapivin, M.; Autaeu, A.; Marchese, M.; et al. 2009 · 2009
Earlier work this paper cites.
Glove: Global vectors for word representation
Pennington, J.; Socher, R.; and Manning, C. D. 2014 · 2014
Earlier work this paper cites.
Word2Vec
Church, K. W. 2017 · 2017
Earlier work this paper cites.
Pointer Sentinel Mixture Models
Merity, S.; Xiong, C.; Bradbury, J.; and Socher, R. 2017 · 2017
Earlier work this paper cites.
Attention is all you need
Vaswani, A.; Shazeer, N.; Parmar, N.; Uszkoreit, J.; Jones, L.; Gomez, A. N.; Kaiser, Ł.; and Polosukhin, I. 2017 · 2017
Earlier work this paper cites.
A discourse-aware attention model for abstractive summarization of long documents
Cohan, A.; Dernoncourt, F.; Kim, D. S.; Bui, T.; Kim, S.; Chang, W.; and Goharian, N. 2018 · 2018
Earlier work this paper cites.
Language models are unsupervised multitask learners
Radford, A.; Wu, J.; Child, R.; Luan, D.; Amodei, D.; Sutskever, I.; et al. 2019 · 2019
Earlier work this paper cites.
Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks
Reimers, N.; and Gurevych, I. 2019 · 2019
Earlier work this paper cites.
Retrieval-augmented generation for knowledge-intensive nlp tasks
Lewis, P.; Perez, E.; Piktus, A.; Petroni, F.; Karpukhin, V.; Goyal, N.; Küttler, H.; Lewis, M.; Yih, W.-t.; Rocktäschel, T.; et al. 2020 · 2020
Earlier work this paper cites.
Summscreen: A dataset for abstractive screenplay summarization
Chen, M.; Chu, Z.; Wiseman, S.; and Gimpel, K. 2021 · 2021
Cited alongside, same era.
Keep It Simple: Unsupervised Simplification of Multi-Paragraph Text
Laban, P.; Schnabel, T.; Bennett, P.; and Hearst, M. A. 2021 · 2021
Cited alongside, same era.
Recurrent memory transformer
Bulatov, A.; Kuratov, Y.; and Burtsev, M. 2022 · 2022
Cited alongside, same era.
A survey on in-context learning
Dong, Q.; Li, L.; Dai, D.; Zheng, C.; Wu, Z.; Chang, B.; Sun, X.; Xu, J.; and Sui, Z. 2022 · 2022
Cited alongside, same era.
Efficient Unsupervised Sentence Compression by Fine-tuning Transformers with Reinforcement Learning
Ghalandari, D. G.; Hokamp, C.; and Ifrim, G. 2022 · 2022
Cited alongside, same era.
Adapting Language Models to Compress Contexts
Chevalier, A.; Wettig, A.; Ajith, A.; and Chen, D. 2023 · 2023
Later among the works it cites.
MeetingBank: A benchmark dataset for meeting summarization
Hu, Y.; Ganter, T.; Deilamsalehy, H.; Dernoncourt, F.; Foroosh, H.; and Liu, F. 2023 · 2023
Later among the works it cites.
ZeroSCROLLS: A Zero-Shot Benchmark for Long Text Understanding
Shaham, U.; Ivgi, M.; Efrat, A.; Berant, J.; and Levy, O. 2023 · 2023
Later among the works it cites.
LLaMA: Open and Efficient Foundation Language Models
Touvron, H.; Lavril, T.; Izacard, G.; Martinet, X.; Lachaux, M.-A.; Lacroix, T.; Rozière, B.; Goyal, N.; Hambro, E.; Azhar, F.; et al. 2023 · 2023
Later among the works it cites.
Llm2vec: Large language models are secretly powerful text encoders
BehnamGhader, P.; Adlakha, V.; Mosbach, M.; Bahdanau, D.; Chapados, N.; and Reddy, S. 2024 · 2024
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
LoRA: Low-Rank Adaptation of Large Language Models
Hu, E. J.; Wallis, P.; Allen-Zhu, Z.; Li, Y.; Wang, S.; Wang, L.; Chen, W.; et al. 2022 · 2022
Cited alongside, same era.
Learned token pruning for transformers
Kim, S.; Shen, S.; Thorsley, D.; Gholami, A.; Kwon, W.; Hassoun, J.; and Keutzer, K. 2022 · 2022
Cited alongside, same era.
Text embeddings by weakly-supervised contrastive pre-training
Wang, L.; Yang, N.; Huang, X.; Jiao, B.; Yang, L.; Jiang, D.; Majumder, R.; and Wei, F. 2022 · 2022
Cited alongside, same era.
Chain-of-thought prompting elicits reasoning in large language models
Wei, J.; Wang, X.; Schuurmans, D.; Bosma, M.; Xia, F.; Chi, E.; Le, Q. V.; Zhou, D.; et al. 2022 · 2022
Cited alongside, same era.
Longbench: A bilingual, multitask benchmark for long context understanding
Bai, Y.; Lv, X.; Zhang, J.; Lyu, H.; Tang, J.; Huang, Z.; Du, Z.; Liu, X.; Zeng, A.; Hou, L.; et al. 2023 · 2023
Cited alongside, same era.
Walking Down the Memory Maze: Beyond Context Limit through Interactive Reading
Chen, H.; Pasunuru, R.; Weston, J. E.; and Celikyilmaz, A. 2023 · 2023
Cited alongside, same era.
Jiang, A. Q.; Sablayrolles, A.; Mensch, A.; Bamford, C.; Chaplot, D. S.; Casas, D. d. l.; Bressand, F.; Lengyel, G.; Lample, G.; Saulnier, L.; et al. 2023a
Cited in the paper.
Closest in time.
In-context Autoencoder for Context Compression in a Large Language Model
Ge, T.; Jing, H.; Wang, L.; Wang, X.; Chen, S.-Q.; and Wei, F. 2024 · 2024
Closest in time.
Discrete prompt compression with reinforcement learning
Jung, H.; and Kim, K.-J. 2024 · 2024
Closest in time.
Llmlingua-2: Data distillation for efficient and faithful task-agnostic prompt compression
Pan, Z.; Wu, Q.; Jiang, H.; Xia, M.; Luo, X.; Zhang, J.; Lin, Q.; Rühle, V.; Yang, Y.; Lin, C.-Y.; et al. 2024 · 2024
Closest in time.
Adapting LLMs for efficient context processing through soft prompt compression
Wang, C.; Yang, Y.; Li, R.; Sun, D.; Cai, R.; Zhang, Y.; Fu, C.; and Floyd, L. 2024 · 2024
Closest in time.
LoMA: Lossless Compressed Memory Attention
Wang, Y.; and Xiao, Z. 2024 · 2024
Closest in time.
RECOMP: Improving retrieval-augmented LMs with context compression and selective augmentation
Xu, F.; Shi, W.; and Choi, E. 2024 · 2024
Closest in time.