LoRASuite: Efficient LoRA Adaptation Across Large Language Model Upgrades

J. Munkres, “Algorithms for the assignment and transportation problems,” Journal of the society for industrial and applied mathematics , vol. 5, no. 1, pp. 32–38, 1957

1957

Earlier work this paper cites.

L. Song, A. Smola, A. Gretton, J. Bedo, and K. Borgwardt, “Feature selection via dependence maximization,” The Journal of Machine Learning Research , vol. 13, pp. 1393–1434, 2012

2012

Earlier work this paper cites.

K. He, X. Zhang, S. Ren, and J. Sun, “Delving deep into rectifiers: Surpassing human-level performance on imagenet classification,” in Proceedings of the IEEE international conference on computer vision , 2015, pp. 1026–1034

2015

Earlier work this paper cites.

R. Koncel-Kedziorski, S. Roy, A. Amini, N. Kushman, and H. Hajishirzi, “Mawps: A math word problem repository,” in Proceedings of the 2016 conference of the north american chapter of the association for computational linguistics: human language technologies , 2016, pp. 1152–1157

2016

Earlier work this paper cites.

W. Ling, D. Yogatama, C. Dyer, and P. Blunsom, “Program induction by rationale generation: Learning to solve and explain algebraic word problems,” arXiv preprint arXiv:1705.04146 , 2017

Original

2017

Earlier work this paper cites.

A. Conneau, G. Kruszewski, G. Lample, L. Barrault, and M. Baroni, “What you can cram into a single vector: Probing sentence embeddings for linguistic properties,” arXiv preprint arXiv:1805.01070 , 2018

Original

2018

Earlier work this paper cites.

M. Yamada, Y. Umezu, K. Fukumizu, and I. Takeuchi, “Post selection inference with kernels,” in International conference on artificial intelligence and statistics . PMLR, 2018, pp. 152–160

2018

Earlier work this paper cites.

P. Clark, I. Cowhey, O. Etzioni, T. Khot, A. Sabharwal, C. Schoenick, and O. Tafjord, “Think you have solved question answering? try arc, the ai2 reasoning challenge,” arXiv preprint arXiv:1803.05457 , 2018

Original

2018

Earlier work this paper cites.

T. Mihaylov, P. Clark, T. Khot, and A. Sabharwal, “Can a suit of armor conduct electricity? a new dataset for open book question answering,” arXiv preprint arXiv:1809.02789 , 2018

Original

2018

Earlier work this paper cites.

A. Morcos, M. Raghu, and S. Bengio, “Insights on representational similarity in neural networks with canonical correlation,” Advances in neural information processing systems , vol. 31, 2018

2018

Earlier work this paper cites.

N. Houlsby, A. Giurgiu, S. Jastrzebski, B. Morrone, Q. De Laroussilhe, A. Gesmundo, M. Attariyan, and S. Gelly, “Parameter-efficient transfer learning for nlp,” in International conference on machine learning . PMLR, 2019, pp. 2790–2799

2019

Earlier work this paper cites.

S. Kornblith, M. Norouzi, H. Lee, and G. Hinton, “Similarity of neural network representations revisited,” in International conference on machine learning . PMLR, 2019, pp. 3519–3529

2019

Earlier work this paper cites.

L. Liu, H. Jiang, P. He, W. Chen, X. Liu, J. Gao, and J. Han, “On the variance of the adaptive learning rate and beyond,” arXiv preprint arXiv:1908.03265 , 2019

Original

2019

Earlier work this paper cites.

C. Clark, K. Lee, M.-W. Chang, T. Kwiatkowski, M. Collins, and K. Toutanova, “Boolq: Exploring the surprising difficulty of natural yes/no questions,” arXiv preprint arXiv:1905.10044 , 2019

Original

2019

Earlier work this paper cites.

M. Sap, H. Rashkin, D. Chen, R. LeBras, and Y. Choi, “Socialiqa: Commonsense reasoning about social interactions,” arXiv preprint arXiv:1904.09728 , 2019

Original

2019

Earlier work this paper cites.

R. Zellers, A. Holtzman, Y. Bisk, A. Farhadi, and Y. Choi, “Hellaswag: Can a machine really finish your sentence?” arXiv preprint arXiv:1905.07830 , 2019

Original

2019

Earlier work this paper cites.

E. Strubell, A. Ganesh, and A. McCallum, “Energy and policy considerations for modern deep learning research,” in Proceedings of the AAAI conference on artificial intelligence , vol. 34, no. 09, 2020, pp. 13 693–13 696

2020

Cited alongside, same era.

Y. Bisk, R. Zellers, J. Gao, Y. Choi et al. , “Piqa: Reasoning about physical commonsense in natural language,” in Proceedings of the AAAI conference on artificial intelligence , vol. 34, no. 05, 2020, pp. 7432–7439

2020

Cited alongside, same era.

B. Lester, R. Al-Rfou, and N. Constant, “The power of scale for parameter-efficient prompt tuning,” arXiv preprint arXiv:2104.08691 , 2021

Original

2021

Cited alongside, same era.

E. B. Zaken, S. Ravfogel, and Y. Goldberg, “Bitfit: Simple parameter-efficient fine-tuning for transformer-based masked language-models,” arXiv preprint arXiv:2106.10199 , 2021

Original

2021

Cited alongside, same era.

A. Conmy, A. Mavor-Parker, A. Lynch, S. Heimersheim, and A. Garriga-Alonso, “Towards automated circuit discovery for mechanistic interpretability,” Advances in Neural Information Processing Systems , vol. 36, pp. 16 318–16 352, 2023

2023

Later among the works it cites.

Q. Zhang, M. Chen, A. Bukharin, P. He, Y. Cheng, W. Chen, and T. Zhao, “Adaptive budget allocation for parameter-efficient fine-tuning,” in The Eleventh International Conference on Learning Representations , 2023. [Online]. Available: https://openreview.net/forum?id=lq62uWRJjiY

2023

Later among the works it cites.

Y. Li, Y. Yu, Q. Zhang, C. Liang, P. He, W. Chen, and T. Zhao, “Losparse: Structured compression of large language models based on low-rank and sparse approximation,” in International Conference on Machine Learning . PMLR, 2023, pp. 20 336–20 350

2023

Later among the works it cites.

Z. Hu, Y. Lan, L. Wang, W. Xu, E.-P. Lim, R. K.-W. Lee, L. Bing, and S. Poria, “Llm-adapters: An adapter family for parameter-efficient fine-tuning of large language models,” arXiv preprint arXiv:2304.01933 , 2023