Fetching the paper…
Reading the bibliography…
Low-Rank Adaptation (LoRA) is a parameter-efficient technique for rapidly fine-tuning foundation models.
A value for n-person games
L. S. Shapley · 1953
Earlier work this paper cites.
Multilinear extensions of games
G. Owen · 1972
Earlier work this paper cites.
Visualizing data using t-sne
L. Van der Maaten and G. Hinton · 2008
Earlier work this paper cites.
Ucf101: A dataset of 101 human actions classes from videos in the wild
K. Soomro · 2012
Earlier work this paper cites.
Describing textures in the wild
M. Cimpoi, S. Maji, I. Kokkinos, S. Mohamed, and A. Vedaldi · 2014
Earlier work this paper cites.
Identifying and attacking the saddle point problem in high-dimensional non-convex optimization
Y. N. Dauphin, R. Pascanu, C. Gulcehre, K. Cho, S. Ganguli, and Y. Bengio · 2014
Earlier work this paper cites.
Explorations on high dimensional landscapes
L. Sagun, V. U. Guney, G. B. Arous, and Y. LeCun · 2014
Earlier work this paper cites.
Deep networks with stochastic depth
G. Huang, Y. Sun, Z. Liu, D. Sedra, and K. Q. Weinberger · 2016
Earlier work this paper cites.
Communication-efficient learning of deep networks from decentralized data
B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. y Arcas · 2017
Earlier work this paper cites.
Attention is all you need
A. Vaswani · 2017
Earlier work this paper cites.
Essentially no barriers in neural network energy landscape
F. Draxler, K. Veschgini, M. Salmhofer, and F. Hamprecht · 2018
Cited alongside, same era.
Gradient descent provably optimizes over-parameterized neural networks
S. S. Du, X. Zhai, B. Poczos, and A. Singh · 2018
Cited alongside, same era.
Loss surfaces, mode connectivity, and fast ensembling of dnns
T. Garipov, P. Izmailov, D. Podoprikhin, D. P. Vetrov, and A. G. Wilson · 2018
Cited alongside, same era.
On lazy training in differentiable programming
L. Chizat, E. Oyallon, and F. Bach · 2019
Cited alongside, same era.
Reducing transformer depth on demand with structured dropout
A. Fan, E. Grave, and A. Joulin · 2020
Cited alongside, same era.
Linear mode connectivity and the lottery ticket hypothesis
J. Frankle, G. K. Dziugaite, D. Roy, and M. Carbin · 2020
Lorahub: Efficient cross-task generalization via dynamic lora composition
C. Huang, Q. Liu, B. Y. Lin, T. Pang, C. Du, and M. Lin · 2023
Later among the works it cites.
Equivariant architectures for learning in deep weight spaces
A. Navon, A. Shamsian, I. Achituve, E. Fetaya, G. Chechik, and H. Maron · 2023
Later among the works it cites.
Progressive prompts: Continual learning for language models
A. Razdaibiedina, Y. Mao, R. Hou, M. Khabsa, M. Lewis, and A. Almahairi · 2023
Later among the works it cites.
Going beyond linear mode connectivity: The layerwise linear feature connectivity
Z. Zhou, Y. Yang, X. Yang, J. Yan, and W. Hu · 2023
Later among the works it cites.
Layer-wise linear mode connectivity
L. Adilova, M. Andriushchenko, M. Kamp, A. Fischer, and M. Jaggi · 2024
Closest in time.
Flora: Federated fine-tuning large language models with heterogeneous low-rank adaptations
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Cited alongside, same era.
A survey on multi-task learning
Y. Zhang and Q. Yang · 2021
Cited alongside, same era.
LoRA: Low-rank adaptation of large language models
E. J. Hu, yelong shen, P. Wallis, Z. Allen-Zhu, Y. Li, S. Wang, L. Wang, and W. Chen · 2022
Cited alongside, same era.
Learning to prompt for vision-language models
K. Zhou, J. Yang, C. C. Loy, and Z. Liu · 2022
Cited alongside, same era.
Git re-basin: Merging models modulo permutation symmetries
S. Ainsworth, J. Hayase, and S. Srinivasa · 2023
Cited alongside, same era.
Z. Wang, Z. Shen, Y. He, G. Sun, H. Wang, L. Lyu, and A. Li · 2024
Closest in time.
Low-rank few-shot adaptation of vision-language models
M. Zanella and I. Ben Ayed · 2024
Closest in time.
Towards building the federatedgpt: Federated instruction tuning
J. Zhang, S. Vahidian, M. Kuo, C. Li, R. Zhang, T. Yu, G. Wang, and Y. Chen · 2024
Closest in time.
Loraretriever: Input-aware lora retrieval and composition for mixed tasks in the wild
Z. Zhao, L. Gan, G. Wang, W. Zhou, H. Yang, K. Kuang, and F. Wu · 2024
Closest in time.
Permutation equivariant neural functionals
A. Zhou, K. Yang, K. Burns, A. Cardace, Y. Jiang, S. Sokota, J. Z. Kolter, and C. Finn · 2024
Closest in time.