Fetching the paper…
Reading the bibliography…
Low-Rank Adaptation (LoRA) is a popular technique for parameter-efficient fine-tuning of Large Language Models (LLMs).
dungeons & dragons , volume 19
G. Gygax and D. Arneson · 1974
Earlier work this paper cites.
The rating of chessplayers: Past and present
A. E. Elo and S. Sloan · 1978
Earlier work this paper cites.
Weight averaging for neural networks and local resampling schemes
J. Utans · 1996
Earlier work this paper cites.
Multitask learning
R. Caruana · 1997
Earlier work this paper cites.
Genetics
V. Singh · 2014
Earlier work this paper cites.
Cross-stitch networks for multi-task learning
I. Misra, A. Shrivastava, A. Gupta, and M. Hebert · 2016
Earlier work this paper cites.
Overcoming catastrophic forgetting in neural networks
J. Kirkpatrick, R. Pascanu, N. Rabinowitz, J. Veness, G. Desjardins, A. A. Rusu, K. Milan, J. Quan, T. Ramalho, A. Grabska-Barwinska, D. Hassabis, C. Clopath, D. Kumaran, and R. Hadsell · 2017
Earlier work this paper cites.
Decoupled weight decay regularization
I. Loshchilov and F. Hutter · 2017
Earlier work this paper cites.
Communication-efficient learning of deep networks from decentralized data
B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. y Arcas · 2017
Earlier work this paper cites.
An investigation of how neural networks learn from the experiences of peers through periodic weight averaging
J. Smith and M. Gashler · 2017
Earlier work this paper cites.
Sampling generative networks, 2017
T. White · 2017
Earlier work this paper cites.
Gradnorm: Gradient normalization for adaptive loss balancing in deep multitask networks
Z. Chen, V. Badrinarayanan, C.-Y. Lee, and A. Rabinovich · 2018
Earlier work this paper cites.
Loss surfaces, mode connectivity, and fast ensembling of dnns
T. Garipov, P. Izmailov, D. Podoprikhin, D. P. Vetrov, and A. G. Wilson · 2018
Earlier work this paper cites.
Averaging weights leads to wider optima and better generalization
P. Izmailov, D. Podoprikhin, T. Garipov, D. Vetrov, and A. G. Wilson · 2018
Earlier work this paper cites.
Modeling task relationships in multi-task learning with multi-gate mixture-of-experts
J. Ma, Z. Zhao, X. Yi, J. Chen, L. Hong, and E. H. Chi · 2018
Earlier work this paper cites.
Know what you don’t know: Unanswerable questions for squad
P. Rajpurkar, R. Jia, and P. Liang · 2018
Earlier work this paper cites.
Multi-task learning as multi-objective optimization
O. Sener and V. Koltun · 2018
Earlier work this paper cites.
Learning to branch for multi-task learning
P. Guo, C.-Y. Lee, and D. Ulbricht · 2020
Earlier work this paper cites.
Stochastic weight averaging in parallel: Large-batch training that generalizes well
V. Gupta, S. A. Serrano, and D. DeCoste · 2020
Earlier work this paper cites.
Adashare: Learning what to share for efficient deep multi-task learning
X. Sun, R. Panda, R. Feris, and K. Saenko · 2020
Earlier work this paper cites.
Training verifiers to solve math word problems
K. Cobbe, V. Kosaraju, M. Bavarian, M. Chen, H. Jun, L. Kaiser, M. Plappert, J. Tworek, J. Hilton, R. Nakano, et al · 2021
Earlier work this paper cites.
Lora: Low-rank adaptation of large language models
E. J. Hu, Y. Shen, P. Wallis, Z. Allen-Zhu, Y. Li, S. Wang, L. Wang, and W. Chen · 2021
Earlier work this paper cites.
Medical terminology for healthcare professions
A. Nelson and K. Greene · 2021
Cited alongside, same era.
Ensemble of averages: Improving model selection and boosting performance in domain generalization
D. Arpit, H. Wang, Y. Zhou, and C. Xiong · 2022
Cited alongside, same era.
Basic cell and molecular biology 5e: What we know and how we find out
G. Bergtrom · 2022
Cited alongside, same era.
Editing models with task arithmetic
G. Ilharco, M. T. Ribeiro, M. Wortsman, S. Gururangan, L. Schmidt, H. Hajishirzi, and A. Farhadi · 2022
Cited alongside, same era.
Branch-train-merge: Embarrassingly parallel training of expert language models
M. Li, S. Gururangan, T. Dettmers, M. Lewis, T. Althoff, N. A. Smith, and L. Zettlemoyer · 2022
Cited alongside, same era.
Ziplora: Any subject in any style by effectively merging loras
V. Shah, N. Ruiz, F. Cole, E. Lu, S. Lazebnik, Y. Li, and V. Jampani · 2023
Later among the works it cites.
Stanford alpaca: An instruction-following llama model
R. Taori, I. Gulrajani, T. Zhang, Y. Dubois, X. Li, C. Guestrin, P. Liang, and T. B. Hashimoto · 2023
Later among the works it cites.
Llama: Open and efficient foundation language models
H. Touvron, T. Lavril, G. Izacard, X. Martinet, M.-A. Lachaux, T. Lacroix, B. Rozière, N. Goyal, E. Hambro, F. Azhar, et al · 2023
Later among the works it cites.
Mole: Mixture of lora experts
X. Wu, S. Huang, and F. Wei · 2023
Later among the works it cites.
TIES-merging: Resolving interference when merging models
P. Yadav, D. Tam, L. Choshen, C. Raffel, and M. Bansal · 2023
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Merging models with fisher-weighted averaging
M. S. Matena and C. A. Raffel · 2022
Cited alongside, same era.
Language models are multilingual chain-of-thought reasoners
F. Shi, M. Suzgun, M. Freitag, X. Wang, S. Srivats, S. Vosoughi, H. W. Chung, Y. Tay, S. Ruder, D. Zhou, et al · 2022
Cited alongside, same era.
Challenging big-bench tasks and whether chain-of-thought can solve them
M. Suzgun, N. Scales, N. Schärli, S. Gehrmann, Y. Tay, H. W. Chung, A. Chowdhery, Q. V. Le, E. H. Chi, D. Zhou, et al · 2022
Cited alongside, same era.
Super-naturalinstructions:generalization via declarative instructions on 1600+ tasks
Y. Wang, S. Mishra, P. Alipoormolabashi, Y. Kordi, A. Mirzaei, A. Arunkumar, A. Ashok, A. S. Dhanasekaran, A. Naik, D. Stap, et al · 2022
Cited alongside, same era.
Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time
M. Wortsman, G. Ilharco, S. Y. Gadre, R. Roelofs, R. Gontijo-Lopes, A. S. Morcos, H. Namkoong, A. Farhadi, Y. Carmon, S. Kornblith, et al · 2022
Cited alongside, same era.
J. Achiam, S. Adler, S. Agarwal, L. Ahmad, I. Akkaya, F. L. Aleman, D. Almeida, J. Altenschmidt, S. Altman, S. Anadkat, et al · 2023
Cited alongside, same era.
Code alpaca: An instruction-following llama model for code generation
S. Chaudhary · 2023
Cited alongside, same era.
E. Yang, Z. Wang, L. Shen, S. Liu, G. Guo, X. Wang, and D. Tao · 2023
Later among the works it cites.
Composing parameter-efficient modules with arithmetic operation
J. Zhang, J. Liu, J. He, et al · 2023
Later among the works it cites.
Evolutionary optimization of model merging recipes, 2024
T. Akiba, M. Shing, Y. Tang, Q. Sun, and D. Ha · 2024
Closest in time.
Mixture-of-loras: An efficient multitask tuning for large language models
W. Feng, C. Hao, Y. Zhang, Y. Han, and H. Wang · 2024
Closest in time.
Arcee’s mergekit: A toolkit for merging large language models, 2024
C. Goddard, S. Siriwardhana, M. Ehghaghi, L. Meyers, V. Karpukhin, B. Benedict, M. McQuade, and J. Solawetz · 2024
Closest in time.
Mix-of-show: Decentralized low-rank adaptation for multi-concept customization of diffusion models
Y. Gu, X. Wang, J. Z. Wu, Y. Shi, Y. Chen, Z. Fan, W. Xiao, R. Zhao, S. Chang, W. Wu, et al · 2024
Closest in time.
Openassistant conversations-democratizing large language model alignment
A. Köpf, Y. Kilcher, D. von Rütte, S. Anagnostidis, Z. R. Tam, K. Stevens, A. Barhoum, D. Nguyen, O. Stanley, R. Nagyfi, et al · 2024
Closest in time.
Leeroo-AI/mergoo
Leroo-AI · 2024
Closest in time.
Moelora: Contrastive learning guided mixture of experts on parameter-efficient fine-tuning for large language models, 2024
T. Luo, J. Lei, F. Lei, W. Liu, S. He, J. Zhao, and K. Liu · 2024
Closest in time.
State of what art? a call for multi-prompt llm evaluation, 2024
M. Mizrahi, G. Kaplan, D. Malkin, R. Dror, D. Shahaf, and G. Stanovsky · 2024
Closest in time.
Learning to route among specialized experts for zero-shot generalization, 2024
M. Muqeeth, H. Liu, Y. Liu, and C. Raffel · 2024
Closest in time.
Quantifying language models’ sensitivity to spurious features in prompt design or: How i learned to start worrying about prompt formatting
M. Sclar, Y. Choi, Y. Tsvetkov, and A. Suhr · 2024
Closest in time.
Pmc-llama: toward building open-source language models for medicine
C. Wu, W. Lin, X. Zhang, Y. Zhang, W. Xie, and Y. Wang · 2024
Closest in time.
Ties-merging: Resolving interference when merging models
P. Yadav, D. Tam, L. Choshen, C. A. Raffel, and M. Bansal · 2024
Closest in time.
Judging llm-as-a-judge with mt-bench and chatbot arena
L. Zheng, W.-L. Chiang, Y. Sheng, S. Zhuang, Z. Wu, Y. Zhuang, Z. Lin, Z. Li, D. Li, E. Xing, et al · 2024
Closest in time.
Multi-lora composition for image generation, 2024
M. Zhong, Y. Shen, S. Wang, Y. Lu, Y. Jiao, S. Ouyang, D. Yu, J. Han, and W. Chen · 2024
Closest in time.
Lima: Less is more for alignment
C. Zhou, P. Liu, P. Xu, S. Iyer, J. Sun, Y. Mao, X. Ma, A. Efrat, P. Yu, L. Yu, et al · 2024
Closest in time.