Fetching the paper…

Reading the bibliography…

2024

Parameter-Efficient Fine-Tuning for Large Models: A Comprehensive Survey

Han, Zeyu, Gao, Chao, Liu, Jinyang et al.

Original
PDF
Post on X

Understand

Large models represent a groundbreaking advancement in multiple application fields, enabling remarkable achievements across various tasks.

However, their unprecedented scale comes with significant computational costs.
These models, often consisting of billions of parameters, require vast amounts of computational resources for execution.
Especially, the expansive scale and computational demands pose considerable challenges when customizing them for particular downstream tasks, particularly over the hardware platforms constrained by computational capabilities.

Parameter-Efficient Fine-Tuning for Large Models: A Comprehensive Survey · Around

Built on

N. Kumari, B. Zhang, R. Zhang, E. Shechtman, and J.-Y. Zhu, “Multi-concept customization of text-to-image diffusion,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2023, pp. 1931–1941
1941
Earlier work this paper cites.
J. Cho, J. Lei, H. Tan, and M. Bansal, “Unifying vision-and-language tasks via text generation,” in International Conference on Machine Learning . PMLR, 2021, pp. 1931–1942
1942
Earlier work this paper cites.
D. J. MacKay, “A practical bayesian framework for backpropagation networks,” Neural computation , vol. 4, no. 3, pp. 448–472, 1992
1992
Earlier work this paper cites.
A. Chambolle, R. A. De Vore, N.-Y. Lee, and B. J. Lucier, “Nonlinear wavelet image processing: variational problems, compression, and noise removal through wavelet shrinkage,” IEEE Transactions on image processing , vol. 7, no. 3, pp. 319–335, 1998
1998
Earlier work this paper cites.
C.-H. Chang, M. Kayed, M. R. Girgis, and K. F. Shaalan, “A survey of web information extraction systems,” IEEE transactions on knowledge and data engineering , vol. 18, no. 10, pp. 1411–1428, 2006
2006
Earlier work this paper cites.
A. Beck and M. Teboulle, “A fast iterative shrinkage-thresholding algorithm for linear inverse problems,” SIAM journal on imaging sciences , vol. 2, no. 1, pp. 183–202, 2009
2009
Earlier work this paper cites.
J. Wright, A. Ganesh, S. Rao, Y. Peng, and Y. Ma, “Robust principal component analysis: Exact recovery of corrupted low-rank matrices via convex optimization,” Advances in neural information processing systems , vol. 22, 2009
2009
Earlier work this paper cites.
M. Everingham, L. Van Gool, C. K. Williams, J. Winn, and A. Zisserman, “The pascal visual object classes (voc) challenge,” International journal of computer vision , vol. 88, pp. 303–338, 2010
2010
Earlier work this paper cites.
A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly et al. , “An image is worth 16x16 words: Transformers for image recognition at scale. arxiv 2020,” arXiv preprint arXiv:2010.11929 , 2010
Original
2010
Earlier work this paper cites.
H. Kuehne, H. Jhuang, E. Garrote, T. Poggio, and T. Serre, “Hmdb: a large video database for human motion recognition,” in 2011 International conference on computer vision . IEEE, 2011, pp. 2556–2563
2011
Earlier work this paper cites.
T. Zhou and D. Tao, “Godec: Randomized low-rank & sparse matrix decomposition in noisy case,” in Proceedings of the 28th International Conference on Machine Learning, ICML 2011 , 2011
2011
Earlier work this paper cites.
T.-Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollár, and C. L. Zitnick, “Microsoft coco: Common objects in context,” in Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V 13 . Springer, 2014, pp. 740–755
2014
Earlier work this paper cites.
I. S. Moreno, P. Garraghan, P. Townend, and J. Xu, “Analysis, modeling and simulation of workload patterns in a large-scale utility cloud,” IEEE Transactions on Cloud Computing , vol. 2, no. 2, pp. 208–221, 2014
2014
Earlier work this paper cites.
S. Antol, A. Agrawal, J. Lu, M. Mitchell, D. Batra, C. L. Zitnick, and D. Parikh, “Vqa: Visual question answering,” in Proceedings of the IEEE international conference on computer vision , 2015, pp. 2425–2433
2015
Earlier work this paper cites.
J. Sohl-Dickstein, E. Weiss, N. Maheswaranathan, and S. Ganguli, “Deep unsupervised learning using nonequilibrium thermodynamics,” in International conference on machine learning . PMLR, 2015, pp. 2256–2265
2015
Earlier work this paper cites.
Q. You, H. Jin, Z. Wang, C. Fang, and J. Luo, “Image captioning with semantic attention,” in Proceedings of the IEEE conference on computer vision and pattern recognition , 2016, pp. 4651–4659
2016
Earlier work this paper cites.
O. Vinyals, A. Toshev, S. Bengio, and D. Erhan, “Show and tell: Lessons learned from the 2015 mscoco image captioning challenge,” IEEE transactions on pattern analysis and machine intelligence , vol. 39, no. 4, pp. 652–663, 2016
2016
Earlier work this paper cites.
A. Dosovitskiy and T. Brox, “Inverting visual representations with convolutional networks,” in Proceedings of the IEEE conference on computer vision and pattern recognition , 2016, pp. 4829–4837
2016
Earlier work this paper cites.
W. Kay, J. Carreira, K. Simonyan, B. Zhang, C. Hillier, S. Vijayanarasimhan, F. Viola, T. Green, T. Back, P. Natsev et al. , “The kinetics human action video dataset,” arXiv preprint arXiv:1705.06950 , 2017
Original
2017
Earlier work this paper cites.
R. Goyal, S. Ebrahimi Kahou, V. Michalski, J. Materzynska, S. Westphal, H. Kim, V. Haenel, I. Fruend, P. Yianilos, M. Mueller-Freitag et al. , “The” something something” video database for learning and evaluating visual common sense,” in Proceedings of the IEEE international conference on computer vision , 2017, pp. 5842–5850
2017
Earlier work this paper cites.
B. Zhou, H. Zhao, X. Puig, S. Fidler, A. Barriuso, and A. Torralba, “Scene parsing through ade20k dataset,” in Proceedings of the IEEE conference on computer vision and pattern recognition , 2017, pp. 633–641
2017
Earlier work this paper cites.
A. N. Gomez, M. Ren, R. Urtasun, and R. B. Grosse, “The reversible residual network: Backpropagation without storing activations,” Advances in neural information processing systems , vol. 30, 2017
2017
Earlier work this paper cites.
S. J. Rennie, E. Marcheret, Y. Mroueh, J. Ross, and V. Goel, “Self-critical sequence training for image captioning,” in Proceedings of the IEEE conference on computer vision and pattern recognition , 2017, pp. 7008–7024
2017
Earlier work this paper cites.
P. Wang, Q. Wu, C. Shen, A. Dick, and A. Van Den Hengel, “Fvqa: Fact-based visual question answering,” IEEE transactions on pattern analysis and machine intelligence , vol. 40, no. 10, pp. 2413–2427, 2017
2017
Earlier work this paper cites.
Q. Wu, D. Teney, P. Wang, C. Shen, A. Dick, and A. Van Den Hengel, “Visual question answering: A survey of methods and datasets,” Computer Vision and Image Understanding , vol. 163, pp. 21–40, 2017
2017
Earlier work this paper cites.
S. Lee, “Toward continual learning for conversational agents,” arXiv preprint arXiv:1712.09943 , 2017
Original
2017
Earlier work this paper cites.
J. Kirkpatrick, R. Pascanu, N. Rabinowitz, J. Veness, G. Desjardins, A. A. Rusu, K. Milan, J. Quan, T. Ramalho, A. Grabska-Barwinska et al. , “Overcoming catastrophic forgetting in neural networks,” Proceedings of the national academy of sciences , vol. 114, no. 13, pp. 3521–3526, 2017
2017
Earlier work this paper cites.
E. Grave, M. M. Cisse, and A. Joulin, “Unbounded cache model for online language modeling with open vocabulary,” Advances in neural information processing systems , vol. 30, 2017
2017
Earlier work this paper cites.
A. Wang, A. Singh, J. Michael, F. Hill, O. Levy, and S. R. Bowman, “Glue: A multi-task benchmark and analysis platform for natural language understanding,” arXiv preprint arXiv:1804.07461 , 2018
Original
2018
Earlier work this paper cites.
T. Mihaylov, P. Clark, T. Khot, and A. Sabharwal, “Can a suit of armor conduct electricity? a new dataset for open book question answering,” in EMNLP , 2018
2018
Earlier work this paper cites.
P. Clark, I. Cowhey, O. Etzioni, T. Khot, A. Sabharwal, C. Schoenick, and O. Tafjord, “Think you have solved question answering? try arc, the ai2 reasoning challenge,” arXiv:1803.05457v1 , 2018
Original
2018
Earlier work this paper cites.
J. Frankle and M. Carbin, “The lottery ticket hypothesis: Finding sparse, trainable neural networks,” arXiv preprint arXiv:1803.03635 , 2018
Original
2018
Earlier work this paper cites.
P. I. Frazier, “A tutorial on bayesian optimization,” arXiv preprint arXiv:1807.02811 , 2018
Original
2018
Earlier work this paper cites.
E. Orhan, “A simple cache model for image recognition,” Advances in Neural Information Processing Systems , vol. 31, 2018
2018
Earlier work this paper cites.
B. Zhang and R. Sennrich, “Root mean square layer normalization,” Advances in Neural Information Processing Systems , vol. 32, 2019
2019
Earlier work this paper cites.
M. Sap, H. Rashkin, D. Chen, R. LeBras, and Y. Choi, “Socialiqa: Commonsense reasoning about social interactions,” arXiv preprint arXiv:1904.09728 , 2019
Original
2019
Earlier work this paper cites.
R. Zellers, A. Holtzman, Y. Bisk, A. Farhadi, and Y. Choi, “Hellaswag: Can a machine really finish your sentence?” in Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics , 2019
2019
Earlier work this paper cites.
C. e. a. Clark, “Boolq: Exploring the surprising difficulty of natural yes/no questions,” in NAACL , 2019
2019
Earlier work this paper cites.
N. Houlsby, A. Giurgiu, S. Jastrzebski, B. Morrone, Q. De Laroussilhe, A. Gesmundo, M. Attariyan, and S. Gelly, “Parameter-efficient transfer learning for nlp,” in International Conference on Machine Learning . PMLR, 2019, pp. 2790–2799
2019
Earlier work this paper cites.
M. Z. Hossain, F. Sohel, M. F. Shiratuddin, and H. Laga, “A comprehensive survey of deep learning for image captioning,” ACM Computing Surveys (CsUR) , vol. 51, no. 6, pp. 1–36, 2019
2019
Earlier work this paper cites.
W. Yang, Y. Xie, A. Lin, X. Li, L. Tan, K. Xiong, M. Li, and J. Lin, “End-to-end open-domain question answering with bertserini,” arXiv preprint arXiv:1902.01718 , 2019
Original
2019
Earlier work this paper cites.
K. Chen, J. Wang, J. Pang, Y. Cao, Y. Xiong, X. Li, S. Sun, W. Feng, Z. Liu, J. Xu, Z. Zhang, D. Cheng, C. Zhu, T. Cheng, Q. Zhao, B. Li, X. Lu, R. Zhu, Y. Wu, J. Dai, J. Wang, J. Shi, W. Ouyang, C. C. Loy, and D. Lin, “MMDetection: Open mmlab detection toolbox and benchmark,” arXiv preprint arXiv:1906.07155 , 2019
Original
2019
Earlier work this paper cites.
Z. He, T. Zhang, and R. B. Lee, “Model inversion attacks against collaborative inference,” in Proceedings of the 35th Annual Computer Security Applications Conference , 2019, pp. 148–162
2019
Earlier work this paper cites.
T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell et al. , “Language models are few-shot learners,” Advances in neural information processing systems , vol. 33, pp. 1877–1901, 2020
2020
Earlier work this paper cites.
Y. Bisk, R. Zellers, R. L. Bras, J. Gao, and Y. Choi, “Piqa: Reasoning about physical commonsense in natural language,” in Thirty-Fourth AAAI Conference on Artificial Intelligence , 2020
2020
Earlier work this paper cites.
J. Pfeiffer, A. Kamath, A. Rücklé, K. Cho, and I. Gurevych, “Adapterfusion: Non-destructive task composition for transfer learning,” arXiv preprint arXiv:2005.00247 , 2020
Original
2020
Earlier work this paper cites.
D. Guo, A. M. Rush, and Y. Kim, “Parameter-efficient transfer learning with diff pruning,” arXiv preprint arXiv:2012.07463 , 2020
Original
2020
Earlier work this paper cites.
A. Aghajanyan, L. Zettlemoyer, and S. Gupta, “Intrinsic dimensionality explains the effectiveness of language model fine-tuning,” arXiv preprint arXiv:2012.13255 , 2020
Original
2020
Earlier work this paper cites.
E. Malach, G. Yehudai, S. Shalev-Schwartz, and O. Shamir, “Proving the lottery ticket hypothesis: Pruning is all you need,” in International Conference on Machine Learning . PMLR, 2020, pp. 6682–6691
2020
Earlier work this paper cites.
J. Liu, A. Moreau, M. Preuss, J. Rapin, B. Roziere, F. Teytaud, and O. Teytaud, “Versatile black-box optimization,” in Proceedings of the 2020 Genetic and Evolutionary Computation Conference , 2020, pp. 620–628
2020
Earlier work this paper cites.
A. Rücklé, G. Geigle, M. Glockner, T. Beck, J. Pfeiffer, N. Reimers, and I. Gurevych, “Adapterdrop: On the efficiency of adapters in transformers,” arXiv preprint arXiv:2010.11918 , 2020
Original
2020
Earlier work this paper cites.
J. O. Zhang, A. Sax, A. Zamir, L. Guibas, and J. Malik, “Side-tuning: a baseline for network adaptation via additive side networks,” in Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part III 16 . Springer, 2020, pp. 698–714
2020
Earlier work this paper cites.
A. Madotto, Z. Lin, Z. Zhou, S. Moon, P. Crook, B. Liu, Z. Yu, E. Cho, and Z. Wang, “Continual learning in task-oriented dialogue systems,” arXiv preprint arXiv:2012.15504 , 2020
Original
2020
Earlier work this paper cites.
J. Ho, A. Jain, and P. Abbeel, “Denoising diffusion probabilistic models,” Advances in neural information processing systems , vol. 33, pp. 6840–6851, 2020
2020
Earlier work this paper cites.
J. Su, Y. Lu, S. Pan, A. Murtadha, B. Wen, and Y. Liu, “Roformer: Enhanced transformer with rotary position embedding,” arXiv preprint arXiv:2104.09864 , 2021
Original
2021
Earlier work this paper cites.
K. Sakaguchi, R. L. Bras, C. Bhagavatula, and Y. Choi, “Winogrande: An adversarial winograd schema challenge at scale,” Communications of the ACM , vol. 64, no. 9, pp. 99–106, 2021
2021
Earlier work this paper cites.
J. He, C. Zhou, X. Ma, T. Berg-Kirkpatrick, and G. Neubig, “Towards a unified view of parameter-efficient transfer learning,” arXiv preprint arXiv:2110.04366 , 2021
Original
2021
Earlier work this paper cites.
Y. Zhu, J. Feng, C. Zhao, M. Wang, and L. Li, “Counter-interference adapter for multilingual machine translation,” arXiv preprint arXiv:2104.08154 , 2021
Original
2021
Earlier work this paper cites.
R. K. Mahabadi, S. Ruder, M. Dehghani, and J. Henderson, “Parameter-efficient multi-task fine-tuning for transformers via shared hypernetworks,” arXiv preprint arXiv:2106.04489 , 2021
Original
2021
Earlier work this paper cites.
X. L. Li and P. Liang, “Prefix-tuning: Optimizing continuous prompts for generation,” arXiv preprint arXiv:2101.00190 , 2021
Original
2021
Earlier work this paper cites.
X. Liu, K. Ji, Y. Fu, W. L. Tam, Z. Du, Z. Yang, and J. Tang, “P-tuning v2: Prompt tuning can be comparable to fine-tuning universally across scales and tasks,” arXiv preprint arXiv:2110.07602 , 2021
Original
2021
Earlier work this paper cites.
X. Liu, Y. Zheng, Z. Du, M. Ding, Y. Qian, Z. Yang, and J. Tang, “Gpt understands, too,” arXiv preprint arXiv:2103.10385 , 2021
Original
2021
Earlier work this paper cites.
B. Lester, R. Al-Rfou, and N. Constant, “The power of scale for parameter-efficient prompt tuning,” arXiv preprint arXiv:2104.08691 , 2021
Original
2021
Earlier work this paper cites.
T. Vu, B. Lester, N. Constant, R. Al-Rfou, and D. Cer, “Spot: Better frozen model adaptation through soft prompt transfer,” arXiv preprint arXiv:2110.07904 , 2021
Original
2021
Earlier work this paper cites.
Y. Su, X. Wang, Y. Qin, C.-M. Chan, Y. Lin, H. Wang, K. Wen, Z. Liu, P. Li, J. Li et al. , “On transferability of prompt tuning for natural language processing,” arXiv preprint arXiv:2111.06719 , 2021
Original
2021
Earlier work this paper cites.
Y. Qin, X. Wang, Y. Su, Y. Lin, N. Ding, J. Yi, W. Chen, Z. Liu, J. Li, L. Hou et al. , “Exploring universal intrinsic task subspace via prompt tuning,” arXiv preprint arXiv:2110.07867 , 2021
Original
2021
Earlier work this paper cites.
Y.-L. Sung, V. Nair, and C. A. Raffel, “Training neural networks with fixed sparse masks,” Advances in Neural Information Processing Systems , vol. 34, pp. 24 193–24 205, 2021
2021
Earlier work this paper cites.
A. Ansell, E. M. Ponti, A. Korhonen, and I. Vulić, “Composable sparse fine-tuning for cross-lingual transfer,” arXiv preprint arXiv:2110.07560 , 2021
Original
2021
Earlier work this paper cites.
R. Xu, F. Luo, Z. Zhang, C. Tan, B. Chang, S. Huang, and F. Huang, “Raise a child in large language model: Towards effective and generalizable fine-tuning,” arXiv preprint arXiv:2109.05687 , 2021
Original
2021
Earlier work this paper cites.
E. B. Zaken, S. Ravfogel, and Y. Goldberg, “Bitfit: Simple parameter-efficient fine-tuning for transformer-based masked language-models,” arXiv preprint arXiv:2106.10199 , 2021
Original
2021
Earlier work this paper cites.
M. Gheini, X. Ren, and J. May, “Cross-attention is all you need: Adapting pretrained transformers for machine translation,” arXiv preprint arXiv:2104.08771 , 2021
Original
2021
Earlier work this paper cites.
E. J. Hu, Y. Shen, P. Wallis, Z. Allen-Zhu, Y. Li, S. Wang, L. Wang, and W. Chen, “Lora: Low-rank adaptation of large language models,” arXiv preprint arXiv:2106.09685 , 2021
Original
2021
Earlier work this paper cites.
R. Karimi Mahabadi, J. Henderson, and S. Ruder, “Compacter: Efficient low-rank hypercomplex adapter layers,” Advances in Neural Information Processing Systems , vol. 34, pp. 1022–1035, 2021
2021
Earlier work this paper cites.
Y. Mao, L. Mathias, R. Hou, A. Almahairi, H. Ma, J. Han, W.-t. Yih, and M. Khabsa, “Unipelt: A unified framework for parameter-efficient language model tuning,” arXiv preprint arXiv:2110.07577 , 2021
Original
2021
Earlier work this paper cites.
M. Chen, H. Peng, J. Fu, and H. Ling, “Autoformer: Searching transformers for visual recognition,” in Proceedings of the IEEE/CVF international conference on computer vision , 2021, pp. 12 270–12 280
2021
Earlier work this paper cites.
A. Steiner, A. Kolesnikov, X. Zhai, R. Wightman, J. Uszkoreit, and L. Beyer, “How to train your vit? data, augmentation, and regularization in vision transformers,” arXiv preprint arXiv:2106.10270 , 2021
Original
2021
Earlier work this paper cites.
X. Chen, S. Xie, and K. He, “An empirical study of training self-supervised vision transformers,” in Proceedings of the IEEE/CVF international conference on computer vision , 2021, pp. 9640–9649
2021
Earlier work this paper cites.
A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark et al. , “Learning transferable visual models from natural language supervision,” in International conference on machine learning . PMLR, 2021, pp. 8748–8763
2021
Earlier work this paper cites.
C. Jia, Y. Yang, Y. Xia, Y.-T. Chen, Z. Parekh, H. Pham, Q. Le, Y.-H. Sung, Z. Li, and T. Duerig, “Scaling up visual and vision-language representation learning with noisy text supervision,” in International conference on machine learning . PMLR, 2021, pp. 4904–4916
2021
Earlier work this paper cites.
Y. Li, F. Liang, L. Zhao, Y. Cui, W. Ouyang, J. Shao, F. Yu, and J. Yan, “Supervision exists everywhere: A data efficient contrastive language-image pre-training paradigm,” arXiv preprint arXiv:2110.05208 , 2021
Original
2021
Earlier work this paper cites.
R. Zhang, R. Fang, W. Zhang, P. Gao, K. Li, J. Dai, Y. Qiao, and H. Li, “Tip-adapter: Training-free clip-adapter for better vision-language modeling,” arXiv preprint arXiv:2111.03930 , 2021
Original
2021
Earlier work this paper cites.
P. Dhariwal and A. Nichol, “Diffusion models beat gans on image synthesis,” Advances in neural information processing systems , vol. 34, pp. 8780–8794, 2021
2021
Earlier work this paper cites.
Y. Wang, S. Mukherjee, X. Liu, J. Gao, A. H. Awadallah, and J. Gao, “Adamix: Mixture-of-adapter for parameter-efficient tuning of large language models,” arXiv preprint arXiv:2205.12410 , vol. 1, no. 2, p. 4, 2022
Original
2022
Earlier work this paper cites.
F. Ma, C. Zhang, L. Ren, J. Wang, Q. Wang, W. Wu, X. Quan, and D. Song, “Xprompt: Exploring the extreme of prompt tuning,” arXiv preprint arXiv:2210.04457 , 2022
Original
2022
Earlier work this paper cites.
Z. Wu, S. Wang, J. Gu, R. Hou, Y. Dong, V. Vydiswaran, and H. Ma, “Idpg: An instance-dependent prompt generation method,” arXiv preprint arXiv:2204.04497 , 2022
Original
2022
Earlier work this paper cites.
X. Liu, T. Sun, X. Huang, and X. Qiu, “Late prompt tuning: A late prompt could be better than many prompts,” arXiv preprint arXiv:2210.11292 , 2022
Original
2022
Earlier work this paper cites.
H. Liu, D. Tam, M. Muqeeth, J. Mohta, T. Huang, M. Bansal, and C. A. Raffel, “Few-shot parameter-efficient fine-tuning is better and cheaper than in-context learning,” Advances in Neural Information Processing Systems , vol. 35, pp. 1950–1965, 2022
2022
Earlier work this paper cites.

Similar

D. Lian, D. Zhou, J. Feng, and X. Wang, “Scaling & shifting your features: A new baseline for efficient model tuning,” Advances in Neural Information Processing Systems , vol. 35, pp. 109–123, 2022
2022
Cited alongside, same era.
D. Vucetic, M. Tayaranian, M. Ziaeefard, J. J. Clark, B. H. Meyer, and W. J. Gross, “Efficient fine-tuning of bert models on the edge,” in 2022 IEEE International Symposium on Circuits and Systems (ISCAS) . IEEE, 2022, pp. 1838–1842
2022
Cited alongside, same era.
A. Edalati, M. Tahaei, I. Kobyzev, V. P. Nia, J. J. Clark, and M. Rezagholizadeh, “Krona: Parameter efficient tuning with kronecker adapter,” arXiv preprint arXiv:2212.10650 , 2022
Original
2022
Cited alongside, same era.
M. Valipour, M. Rezagholizadeh, I. Kobyzev, and A. Ghodsi, “Dylora: Parameter efficient tuning of pre-trained models using dynamic search-free low-rank adaptation,” arXiv preprint arXiv:2210.07558 , 2022
Original
2022
Cited alongside, same era.

Then

Y. Li, Y. Yu, C. Liang, P. He, N. Karampatziakis, W. Chen, and T. Zhao, “Loftq: Lora-fine-tuning-aware quantization for large language models,” arXiv preprint arXiv:2310.08659 , 2023
Original
2023
Later among the works it cites.
H. Guo, P. Greengard, E. P. Xing, and Y. Kim, “Lq-lora: Low-rank plus quantized matrix decomposition for efficient language model finetuning,” arXiv preprint arXiv:2311.12023 , 2023
Original
2023
Later among the works it cites.
Y. Xu, L. Xie, X. Gu, X. Chen, H. Chang, H. Zhang, Z. Chen, X. Zhang, and Q. Tian, “Qa-lora: Quantization-aware low-rank adaptation of large language models,” arXiv preprint arXiv:2309.14717 , 2023
Original
2023
Later among the works it cites.
Y. Chai, J. Gkountouras, G. G. Ko, D. Brooks, and G.-Y. Wei, “Int2. 1: Towards fine-tunable quantized large language models with error correction through low-rank adaptation,” arXiv preprint arXiv:2306.08162 , 2023
Original
2023
Later among the works it cites.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Y. Zhang, K. Zhou, and Z. Liu, “Neural prompt search,” 2022

2022

Cited alongside, same era.

S. Hu, Z. Zhang, N. Ding, Y. Wang, Y. Wang, Z. Liu, and M. Sun, “Sparse structure search for parameter-efficient tuning,” arXiv preprint arXiv:2206.07382 , 2022

2022

Cited alongside, same era.

H. Wu and X. Shi, “Adversarial soft prompt tuning for cross-domain sentiment analysis,” in Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , 2022, pp. 2438–2447

2022

Cited alongside, same era.

J. Antorán, D. Janz, J. U. Allingham, E. Daxberger, R. R. Barbano, E. Nalisnick, and J. M. Hernández-Lobato, “Adapting the linearised laplace model evidence for modern deep learning,” in International Conference on Machine Learning . PMLR, 2022, pp. 796–821

2022

Cited alongside, same era.

S. He, L. Ding, D. Dong, J. Zhang, and D. Tao, “SparseAdapter: An easy approach for improving the parameter-efficiency of adapters,” in Findings of the Association for Computational Linguistics: EMNLP 2022 . Abu Dhabi, United Arab Emirates: Association for Computational Linguistics, Dec. 2022, pp. 2184–2190. [Online]. Available: https://aclanthology.org/2022.findings-emnlp.160

2022

Cited alongside, same era.

L. Hedegaard, A. Alok, J. Jose, and A. Iosifidis, “Structured pruning adapters,” arXiv preprint arXiv:2211.10155 , 2022

2022

Cited alongside, same era.

Y.-L. Sung, J. Cho, and M. Bansal, “Lst: Ladder side-tuning for parameter and memory efficient transfer learning,” Advances in Neural Information Processing Systems , vol. 35, pp. 12 991–13 005, 2022

2022

Cited alongside, same era.

Y.-L. Sung, J. Cho, and M. Bansal, “Vl-adapter: Parameter-efficient transfer learning for vision-and-language tasks,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2022, pp. 5227–5237

2022

Cited alongside, same era.

Z. Jiang, C. Mao, Z. Huang, A. Ma, Y. Lv, Y. Shen, D. Zhao, and J. Zhou, “Res-tuning: A flexible and efficient tuning paradigm via unbinding tuner from backbone,” arXiv preprint arXiv:2310.19859 , 2023

2023

Later among the works it cites.

B. Liao, S. Tan, and C. Monz, “Make your pre-trained model reversible: From parameter to memory efficient fine-tuning,” arXiv preprint arXiv:2306.00477 , 2023

2023

Later among the works it cites.

L. Zhang, L. Zhang, S. Shi, X. Chu, and B. Li, “Lora-fa: Memory-efficient low-rank adaptation for large language models fine-tuning,” arXiv preprint arXiv:2308.03303 , 2023

2023

Later among the works it cites.

J. Phang, Y. Mao, P. He, and W. Chen, “Hypertuning: Toward adapting large language models without back-propagation,” in International Conference on Machine Learning . PMLR, 2023, pp. 27 854–27 875

2023

Later among the works it cites.

F. Jin, J. Zhang, and C. Zong, “Parameter-efficient tuning for large language model without calculating its gradients,” in Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing , 2023, pp. 321–330

2023

Later among the works it cites.

S. Malladi, T. Gao, E. Nichani, A. Damian, J. D. Lee, D. Chen, and S. Arora, “Fine-tuning language models with just forward passes,” arXiv preprint arXiv:2305.17333 , 2023

2023

Later among the works it cites.

Y. Sheng, S. Cao, D. Li, C. Hooper, N. Lee, S. Yang, C. Chou, B. Zhu, L. Zheng, K. Keutzer et al. , “S-lora: Serving thousands of concurrent lora adapters,” arXiv preprint arXiv:2311.03285 , 2023

2023

Later among the works it cites.

Y. Huang, Y. Li, Y. Xu, L. Zhang, R. Gan, J. Zhang, and L. Wang, “Mvp-tuning: Multi-view knowledge retrieval with prompt tuning for commonsense reasoning,” in Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , 2023, pp. 13 417–13 432

2023

Later among the works it cites.

Z. Zhao, L. Hu, H. Zhao, Y. Shao, and Y. Wang, “Knowledgeable parameter efficient tuning network for commonsense question answering,” in Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , 2023, pp. 9051–9063

2023

Later among the works it cites.

H. Zhao, R. He, M. Xiao, and J. Xu, “Infusing hierarchical guidance into prompt tuning: A parameter-efficient framework for multi-level implicit discourse relation recognition,” in Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , 2023, pp. 6477–6492

2023

Later among the works it cites.

Y. Ouyang, Y. Cao, Y. Gao, Z. Wu, J. Zhang, and X. Dai, “On prefix-tuning for lightweight out-of-distribution detection,” in Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , 2023, pp. 1533–1545

2023

Later among the works it cites.

M. S. Ozdayi, C. Peris, J. Fitzgerald, C. Dupuy, J. Majmudar, H. Khan, R. Parikh, and R. Gupta, “Controlling the extraction of memorized data from large language models via prompt-tuning,” arXiv preprint arXiv:2305.11759 , 2023

2023

Later among the works it cites.

G. Xiao, J. Lin, and S. Han, “Offsite-tuning: Transfer learning without full model,” arXiv preprint arXiv:2302.04870 , 2023

2023

Later among the works it cites.

T. Che, J. Liu, Y. Zhou, J. Ren, J. Zhou, V. S. Sheng, H. Dai, and D. Dou, “Federated learning of large language models with parameter-efficient prompt tuning and adaptive optimization,” arXiv preprint arXiv:2310.15080 , 2023

2023

Later among the works it cites.

Y. Li, M. Du, X. Wang, and Y. Wang, “Prompt tuning pushes farther, contrastive learning pulls closer: A two-stage approach to mitigate social biases,” arXiv preprint arXiv:2307.01595 , 2023

Original

2023

Later among the works it cites.

D. Zhu, J. Chen, X. Shen, X. Li, and M. Elhoseiny, “Minigpt-4: Enhancing vision-language understanding with advanced large language models,” arXiv preprint arXiv:2304.10592 , 2023

2023

Later among the works it cites.

H. Liu, C. Li, Q. Wu, and Y. J. Lee, “Visual instruction tuning,” arXiv preprint arXiv:2304.08485 , 2023

2023

Later among the works it cites.

Z.-Y. Hu, Y. Li, M. R. Lyu, and L. Wang, “Vl-pet: Vision-and-language parameter-efficient tuning via granularity control,” in Proceedings of the IEEE/CVF International Conference on Computer Vision , 2023, pp. 3010–3020

2023

Later among the works it cites.

R. Zhang, J. Han, A. Zhou, X. Hu, S. Yan, P. Lu, H. Li, P. Gao, and Y. Qiao, “Llama-adapter: Efficient fine-tuning of language models with zero-init attention,” arXiv preprint arXiv:2303.16199 , 2023

2023

Later among the works it cites.

P. Gao, J. Han, R. Zhang, Z. Lin, S. Geng, A. Zhou, W. Zhang, P. Lu, C. He, X. Yue et al. , “Llama-adapter v2: Parameter-efficient visual instruction model,” arXiv preprint arXiv:2304.15010 , 2023

2023

Later among the works it cites.

B. Zhao, H. Tu, C. Wei, J. Mei, and C. Xie, “Tuning layernorm in attention: Towards efficient multi-modal llm finetuning,” arXiv preprint arXiv:2312.11420 , 2023

2023

Later among the works it cites.

Z. Liang, F. Wei, Y. Jie, Y. Qian, Z. Hao, and B. Han, “Prompts can play lottery tickets well: Achieving lifelong information extraction via lottery prompt tuning,” in Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , 2023, pp. 277–292

2023

Later among the works it cites.

X. Wang, T. Chen, Q. Ge, H. Xia, R. Bao, R. Zheng, Q. Zhang, T. Gui, and X. Huang, “Orthogonal subspace learning for language model continual learning,” arXiv preprint arXiv:2310.14152 , 2023

2023

Later among the works it cites.

S. Chen, S. Wong, L. Chen, and Y. Tian, “Extending context window of large language models via positional interpolation,” arXiv preprint arXiv:2306.15595 , 2023

2023

Later among the works it cites.

Y. Chen, S. Qian, H. Tang, X. Lai, Z. Liu, S. Han, and J. Jia, “Longlora: Efficient fine-tuning of long-context large language models,” arXiv preprint arXiv:2309.12307 , 2023

2023

Later among the works it cites.

J. Yang, “Longqlora: Efficient and effective method to extend context length of large language models,” arXiv preprint arXiv:2311.04879 , 2023

2023

Later among the works it cites.

Y. Sheng, L. Zheng, B. Yuan, Z. Li, M. Ryabinin, B. Chen, P. Liang, C. Ré, I. Stoica, and C. Zhang, “Flexgen: High-throughput generative inference of large language models with a single gpu,” in International Conference on Machine Learning . PMLR, 2023, pp. 31 094–31 116

2023

Later among the works it cites.

M. Dehghani, J. Djolonga, B. Mustafa, P. Padlewski, J. Heek, J. Gilmer, A. P. Steiner, M. Caron, R. Geirhos, I. Alabdulmohsin et al. , “Scaling vision transformers to 22 billion parameters,” in International Conference on Machine Learning . PMLR, 2023, pp. 7480–7512

2023

Later among the works it cites.

Q. Gao, C. Zhao, Y. Sun, T. Xi, G. Zhang, B. Ghanem, and J. Zhang, “A unified continual learning framework with general parameter-efficient tuning,” arXiv preprint arXiv:2303.10070 , 2023

2023

Later among the works it cites.

S. Yoo, E. Kim, D. Jung, J. Lee, and S. Yoon, “Improving visual prompt tuning for self-supervised vision transformers,” arXiv preprint arXiv:2306.05067 , 2023

2023

Later among the works it cites.

T. Yang, Y. Zhu, Y. Xie, A. Zhang, C. Chen, and M. Li, “Aim: Adapting image models for efficient video action recognition,” arXiv preprint arXiv:2302.03024 , 2023

2023

Later among the works it cites.

M. Xu, Z. Zhang, F. Wei, H. Hu, and X. Bai, “Side adapter network for open-vocabulary semantic segmentation,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2023, pp. 2945–2954

2023

Later among the works it cites.

Q. Yu, J. He, X. Deng, X. Shen, and L.-C. Chen, “Convolutions die hard: Open-vocabulary segmentation with single frozen convolutional clip,” arXiv preprint arXiv:2308.02487 , 2023

2023

Later among the works it cites.

Z. Xu, Z. Chen, Y. Zhang, Y. Song, X. Wan, and G. Li, “Bridging vision and language encoders: Parameter-efficient tuning for referring image segmentation,” in Proceedings of the IEEE/CVF International Conference on Computer Vision , 2023, pp. 17 503–17 512

2023

Later among the works it cites.

X. Zhu, R. Zhang, B. He, Z. Guo, Z. Zeng, Z. Qin, S. Zhang, and P. Gao, “Pointclip v2: Prompting clip and gpt for powerful 3d open-world learning,” in Proceedings of the IEEE/CVF International Conference on Computer Vision , 2023, pp. 2639–2650

2023

Later among the works it cites.

T. Huang, B. Dong, Y. Yang, X. Huang, R. W. Lau, W. Ouyang, and W. Zuo, “Clip2point: Transfer clip to point cloud classification with image-depth pre-training,” in Proceedings of the IEEE/CVF International Conference on Computer Vision , 2023, pp. 22 157–22 167

2023

Later among the works it cites.

Z. Han, F. Zhu, Q. Lao, and H. Jiang, “Zero-shot referring expression comprehension via structural similarity between images and captions,” arXiv preprint arXiv:2311.17048 , 2023

2023

Later among the works it cites.

S. Doveh, A. Arbelle, S. Harary, E. Schwartz, R. Herzig, R. Giryes, R. Feris, R. Panda, S. Ullman, and L. Karlinsky, “Teaching structured vision & language concepts to vision & language models,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2023, pp. 2657–2668

2023

Later among the works it cites.

B. Zhu, Y. Niu, Y. Han, Y. Wu, and H. Zhang, “Prompt-aligned gradient for prompt tuning,” in Proceedings of the IEEE/CVF International Conference on Computer Vision , 2023, pp. 15 659–15 669

2023

Later among the works it cites.

M. U. Khattak, H. Rasheed, M. Maaz, S. Khan, and F. S. Khan, “Maple: Multi-modal prompt learning,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2023, pp. 19 113–19 122

2023

Later among the works it cites.

C.-M. Feng, K. Yu, Y. Liu, S. Khan, and W. Zuo, “Diverse data augmentation with diffusions for effective test-time prompt tuning,” in Proceedings of the IEEE/CVF International Conference on Computer Vision , 2023, pp. 2704–2714

2023

Later among the works it cites.

P. Gao, S. Geng, R. Zhang, T. Ma, R. Fang, Y. Zhang, H. Li, and Y. Qiao, “Clip-adapter: Better vision-language models with feature adapters,” International Journal of Computer Vision , pp. 1–15, 2023

2023

Later among the works it cites.

Z. Han, Y. Wang, L. Zhou, P. Wang, B. Yan, J. Zhou, Y. Wang, and D. Shen, “Contrastive diffusion model with auxiliary guidance for coarse-to-fine pet reconstruction,” in International Conference on Medical Image Computing and Computer-Assisted Intervention . Springer, 2023, pp. 239–249

2023

Later among the works it cites.

L. Yang, Z. Zhang, Y. Song, S. Hong, R. Xu, Y. Zhao, W. Zhang, B. Cui, and M.-H. Yang, “Diffusion models: A comprehensive survey of methods and applications,” ACM Computing Surveys , vol. 56, no. 4, pp. 1–39, 2023

2023

Later among the works it cites.

F.-A. Croitoru, V. Hondru, R. T. Ionescu, and M. Shah, “Diffusion models in vision: A survey,” IEEE Transactions on Pattern Analysis and Machine Intelligence , 2023

2023

Later among the works it cites.

N. Ruiz, Y. Li, V. Jampani, Y. Pritch, M. Rubinstein, and K. Aberman, “Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2023, pp. 22 500–22 510

2023

Later among the works it cites.

S. Luo, Y. Tan, S. Patil, D. Gu, P. von Platen, A. Passos, L. Huang, J. Li, and H. Zhao, “Lcm-lora: A universal stable-diffusion acceleration module,” arXiv preprint arXiv:2311.05556 , 2023

2023

Later among the works it cites.

W. Chai, D. Zheng, J. Cao, Z. Chen, C. Wang, and C. Ma, “Speedupnet: A plug-and-play hyper-network for accelerating text-to-image diffusion models,” arXiv preprint arXiv:2312.08887 , 2023

2023

Later among the works it cites.

J. Z. Wu, Y. Ge, X. Wang, S. W. Lei, Y. Gu, Y. Shi, W. Hsu, Y. Shan, X. Qie, and M. Z. Shou, “Tune-a-video: One-shot tuning of image diffusion models for text-to-video generation,” in Proceedings of the IEEE/CVF International Conference on Computer Vision , 2023, pp. 7623–7633

2023

Later among the works it cites.

Z. Xing, Q. Dai, H. Hu, Z. Wu, and Y.-G. Jiang, “Simda: Simple diffusion adapter for efficient video generation,” arXiv preprint arXiv:2308.09710 , 2023

2023

Later among the works it cites.

B. Zeng, S. Li, Y. Feng, H. Li, S. Gao, J. Liu, H. Li, X. Tang, J. Liu, and B. Zhang, “Ipdreamer: Appearance-controllable 3d object generation with image prompts,” arXiv preprint arXiv:2310.05375 , 2023

2023

Later among the works it cites.

L. Zhang, A. Rao, and M. Agrawala, “Adding conditional control to text-to-image diffusion models,” in Proceedings of the IEEE/CVF International Conference on Computer Vision , 2023, pp. 3836–3847

2023

Later among the works it cites.

R. Gandikota, J. Materzynska, T. Zhou, A. Torralba, and D. Bau, “Concept sliders: Lora adaptors for precise control in diffusion models,” arXiv preprint arXiv:2311.12092 , 2023

2023

Later among the works it cites.

C. Mou, X. Wang, L. Xie, Y. Wu, J. Zhang, Z. Qi, Y. Shan, and X. Qie, “T2i-adapter: Learning adapters to dig out more controllable ability for text-to-image diffusion models,” arXiv preprint arXiv:2302.08453 , 2023

2023

Later among the works it cites.

H. Ye, J. Zhang, S. Liu, X. Han, and W. Yang, “Ip-adapter: Text compatible image prompt adapter for text-to-image diffusion models,” arXiv preprint arXiv:2308.06721 , 2023

2023

Later among the works it cites.

OpenAI, “Gpt-4,” in https://openai.com/gpt-4 , 2023

2023

Later among the works it cites.

G. Team, R. Anil, S. Borgeaud, Y. Wu, J.-B. Alayrac, J. Yu, R. Soricut, J. Schalkwyk, A. M. Dai, A. Hauth et al. , “Gemini: a family of highly capable multimodal models,” arXiv preprint arXiv:2312.11805 , 2023

2023

Later among the works it cites.

G. Xiao, J. Lin, and S. Han, “Offsite-tuning: Transfer learning without full model,” arXiv preprint arXiv:2302.04870 , 2023

2023

Later among the works it cites.

L. Chen, Z. Ye, Y. Wu, D. Zhuo, L. Ceze, and A. Krishnamurthy, “Punica: Multi-tenant lora serving,” arXiv preprint arXiv:2310.18547 , 2023

2023

Later among the works it cites.

C. Poth, H. Sterz, I. Paul, S. Purkayastha, L. Engländer, T. Imhof, I. Vulić, S. Ruder, I. Gurevych, and J. Pfeiffer, “Adapters: A unified library for parameter-efficient and modular transfer learning,” 2023

2023

Later among the works it cites.

S. Q. Zhang, T. Tambe, N. Cuevas, G.-Y. Wei, and D. Brooks, “Camel: Co-designing ai models and embedded drams for efficient on-device learning,” arXiv preprint arXiv:2305.03148 , 2023

2023

Later among the works it cites.

A. Gu and T. Dao, “Mamba: Linear-time sequence modeling with selective state spaces,” arXiv preprint arXiv:2312.00752 , 2023

2023

Later among the works it cites.

Y. Bai, X. Geng, K. Mangalam, A. Bar, A. Yuille, T. Darrell, J. Malik, and A. A. Efros, “Sequential modeling enables scalable learning for large vision models,” arXiv preprint arXiv:2312.00785 , 2023

Original

2023

Later among the works it cites.

S.-Y. Liu, C.-Y. Wang, H. Yin, P. Molchanov, Y.-C. F. Wang, K.-T. Cheng, and M.-H. Chen, “Dora: Weight-decomposed low-rank adaptation,” arXiv preprint arXiv:2402.09353 , 2024

2024

Closest in time.

R. Zhang, R. Qiang, S. A. Somayajula, and P. Xie, “Autolora: Automatically tuning matrix ranks in low-rank adaptation based on meta learning,” arXiv preprint arXiv:2403.09113 , 2024

2024

Closest in time.

Y. Lin, X. Ma, X. Chu, Y. Jin, Z. Yang, Y. Wang, and H. Mei, “Lora dropout as a sparsity regularizer for overfitting control,” arXiv preprint arXiv:2404.09610 , 2024

2024

Closest in time.

X. Meng, D. Dai, W. Luo, Z. Yang, S. Wu, X. Wang, P. Wang, Q. Dong, L. Chen, and Z. Sui, “Periodiclora: Breaking the low-rank bottleneck in lora optimization,” arXiv preprint arXiv:2402.16141 , 2024

2024

Closest in time.

S. Hayou, N. Ghosh, and B. Yu, “Lora+: Efficient low rank adaptation of large models,” arXiv preprint arXiv:2402.12354 , 2024

2024

Closest in time.

T. Wu, J. Wang, Z. Zhao, and N. Wong, “Mixture-of-subspaces in low-rank adaptation,” arXiv preprint arXiv:2406.11909 , 2024

2024

Closest in time.

W. Feng, C. Hao, Y. Zhang, Y. Han, and H. Wang, “Mixture-of-loras: An efficient multitask tuning for large language models,” arXiv preprint arXiv:2403.03432 , 2024

2024

Closest in time.

X. Wu, S. Huang, and F. Wei, “Mixture of lora experts,” arXiv preprint arXiv:2404.13628 , 2024

2024

Closest in time.

D. Li, Y. Ma, N. Wang, Z. Cheng, L. Duan, J. Zuo, C. Yang, and M. Tang, “Mixlora: Enhancing large language models fine-tuning with lora based mixture of experts,” arXiv preprint arXiv:2404.15159 , 2024

2024

Closest in time.

V. Fomenko, H. Yu, J. Lee, S. Hsieh, and W. Chen, “A note on lora,” arXiv preprint arXiv:2404.05086 , 2024

2024

Closest in time.

H. Rajabzadeh, M. Valipour, T. Zhu, M. Tahaei, H. J. Kwon, A. Ghodsi, B. Chen, and M. Rezagholizadeh, “Qdylora: Quantized dynamic low-rank adaptation for efficient large language model tuning,” arXiv preprint arXiv:2402.10462 , 2024

2024

Closest in time.

J. Liu, G. Xiao, K. Li, J. D. Lee, S. Han, T. Dao, and T. Cai, “Bitdelta: Your fine-tune may only be worth one bit,” arXiv preprint arXiv:2402.10193 , 2024

2024

Closest in time.

J. Zhao, Z. Zhang, B. Chen, Z. Wang, A. Anandkumar, and Y. Tian, “Galore: Memory-efficient llm training by gradient low-rank projection,” arXiv preprint arXiv:2403.03507 , 2024

2024

Closest in time.

S. Tan, X. Li, S. Patil, Z. Wu, T. Zhang, K. Keutzer, J. E. Gonzalez, and R. A. Popa, “Lloco: Learning long contexts offline,” arXiv preprint arXiv:2404.07979 , 2024

2024

Closest in time.

Z. Zhang, Y. Sheng, T. Zhou, T. Chen, L. Zheng, R. Cai, Z. Song, Y. Tian, C. Ré, C. Barrett et al. , “H2o: Heavy-hitter oracle for efficient generative inference of large language models,” Advances in Neural Information Processing Systems , vol. 36, 2024

2024

Closest in time.

H. Kang, Q. Zhang, S. Kundu, G. Jeong, Z. Liu, T. Krishna, and T. Zhao, “Gear: An efficient kv cache compression recipefor near-lossless generative inference of llm,” arXiv preprint arXiv:2403.05527 , 2024

Original

2024

Closest in time.

L. Ren, C. Chen, L. Wang, and K. Hua, “Learning semantic proxies from visual prompts for parameter-efficient fine-tuning in deep metric learning,” arXiv preprint arXiv:2402.02340 , 2024

Original

2024

Closest in time.

C. Gao and S. Q. Zhang, “Dlora: Distributed parameter-efficient fine-tuning solution for large language model,” arXiv preprint arXiv:2404.05182 , 2024

2024

Closest in time.

T. Brooks, B. Peebles, C. Holmes, W. DePue, Y. Guo, L. Jing, D. Schnurr, J. Taylor, T. Luhman, E. Luhman, C. Ng, R. Wang, and A. Ramesh, “Video generation models as world simulators,” 2024. [Online]. Available: https://openai.com/research/video-generation-models-as-world-simulators

2024

Closest in time.