TeamLoRA: Boosting Low-Rank Adaptation with Expert Collaboration and Competition

A value for n-person games

Shapley, L. S.; et al. 1953 · 1953

Earlier work this paper cites.

Few-shot parameter-efficient fine-tuning is better and cheaper than in-context learning

Liu, H.; Tam, D.; Muqeeth, M.; Mohta, J.; Huang, T.; Bansal, M.; and Raffel, C. A. 2022 · 1965

Earlier work this paper cites.

Adaptive mixtures of local experts

Jacobs, R. A.; Jordan, M. I.; Nowlan, S. J.; and Hinton, G. E. 1991 · 1991

Earlier work this paper cites.

Gshard: Scaling giant models with conditional computation and automatic sharding

Original

Lepikhin, D.; Lee, H.; Xu, Y.; Chen, D.; Firat, O.; Huang, Y.; Krikun, M.; Shazeer, N.; and Chen, Z. 2020 · 2006

Earlier work this paper cites.

Learning Word Vectors for Sentiment Analysis

Maas, A. L.; Daly, R. E.; Pham, P. T.; Huang, D.; Ng, A. Y.; and Potts, C. 2011 · 2011

Earlier work this paper cites.

Dataset and Neural Recurrent Sequence Labeling Model for Open-Domain Factoid Question Answering

Original

Li, P.; Li, W.; He, Z.; Wang, X.; Cao, Y.; Zhou, J.; and Xu, W. 2016 · 2016

Earlier work this paper cites.

triviaqa: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension

Original

Joshi, M.; Choi, E.; Weld, D.; and Zettlemoyer, L. 2017 · 2017

Earlier work this paper cites.

Outrageously large neural networks: The sparsely-gated mixture-of-experts layer

Original

Shazeer, N.; Mirhoseini, A.; Maziarz, K.; Davis, A.; Le, Q.; Hinton, G.; and Dean, J. 2017 · 2017

Earlier work this paper cites.

Bilateral multi-perspective matching for natural language sentences

Original

Wang, Z.; Hamza, W.; and Florian, R. 2017 · 2017

Earlier work this paper cites.

Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge

Original

Clark, P.; Cowhey, I.; Etzioni, O.; Khot, T.; Sabharwal, A.; Schoenick, C.; and Tafjord, O. 2018 · 2018

Earlier work this paper cites.

Vizwiz grand challenge: Answering visual questions from blind people

Gurari, D.; Li, Q.; Stangl, A. J.; Guo, A.; Lin, C.; Grauman, K.; Luo, J.; and Bigham, J. P. 2018 · 2018

Earlier work this paper cites.

Parameter-efficient transfer learning for NLP

Houlsby, N.; Giurgiu, A.; Jastrzebski, S.; Morrone, B.; De Laroussilhe, Q.; Gesmundo, A.; Attariyan, M.; and Gelly, S. 2019 · 2019

Earlier work this paper cites.

Natural questions: a benchmark for question answering research

Kwiatkowski, T.; Palomaki, J.; Redfield, O.; Collins, M.; Parikh, A.; Alberti, C.; Epstein, D.; Polosukhin, I.; Devlin, J.; Lee, K.; et al. 2019 · 2019

Earlier work this paper cites.

Towards vqa models that can read

Singh, A.; Natarajan, V.; Shah, M.; Jiang, Y.; Chen, X.; Batra, D.; Parikh, D.; and Rohrbach, M. 2019 · 2019

Earlier work this paper cites.

GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding

Original

Wang, A.; Singh, A.; Michael, J.; Hill, F.; Levy, O.; and Bowman, S. R. 2019 · 2019

Earlier work this paper cites.

Adversarial NLI: A New Benchmark for Natural Language Understanding

Nie, Y.; Williams, A.; Dinan, E.; Bansal, M.; Weston, J.; and Kiela, D. 2020 · 2020

Earlier work this paper cites.

Learning to summarize with human feedback

Stiennon, N.; Ouyang, L.; Wu, J.; Ziegler, D.; Lowe, R.; Voss, C.; Radford, A.; Amodei, D.; and Christiano, P. F. 2020 · 2020

Earlier work this paper cites.

Towards a unified view of parameter-efficient transfer learning

Original

He, J.; Zhou, C.; Ma, X.; Berg-Kirkpatrick, T.; and Neubig, G. 2021 · 2021

Earlier work this paper cites.

Measuring Massive Multitask Language Understanding

Hendrycks, D.; Burns, C.; Basart, S.; Zou, A.; Mazeika, M.; Song, D.; and Steinhardt, J. 2021 · 2021

Earlier work this paper cites.

The Power of Scale for Parameter-Efficient Prompt Tuning

Original

Lester, B.; Al-Rfou, R.; and Constant, N. 2021 · 2021

Earlier work this paper cites.

Prefix-tuning: Optimizing continuous prompts for generation

Original

Li, X. L.; and Liang, P. 2021 · 2021

Cited alongside, same era.

P-tuning v2: Prompt tuning can be comparable to fine-tuning universally across scales and tasks

Original

Liu, X.; Ji, K.; Fu, Y.; Tam, W. L.; Du, Z.; Yang, Z.; and Tang, J. 2021 · 2021

Cited alongside, same era.

Learning transferable visual models from natural language supervision

Radford, A.; Kim, J. W.; Hallacy, C.; Ramesh, A.; Goh, G.; Agarwal, S.; Sastry, G.; Askell, A.; Mishkin, P.; Clark, J.; et al. 2021 · 2021

Cited alongside, same era.

Winogrande: An adversarial winograd schema challenge at scale

Sakaguchi, K.; Bras, R. L.; Bhagavatula, C.; and Choi, Y. 2021 · 2021

Cited alongside, same era.

Taming sparsely activated transformer with stochastic experts

Original

Zuo, S.; Liu, X.; Jiao, J.; Kim, Y. J.; Hassan, H.; Zhang, R.; Zhao, T.; and Gao, J. 2021 · 2021

Internlm2 technical report

Original

Cai, Z.; Cao, M.; Chen, H.; Chen, K.; Chen, K.; Chen, X.; Chen, X.; Chen, Z.; Chen, Z.; Chu, P.; et al. 2024 · 2024

Closest in time.

The Llama 3 Herd of Models

Original

Dubey, A.; Jauhri, A.; Pandey, A.; Kadian, A.; Al-Dahle, A.; Letman, A.; Mathur, A.; Schelten, A.; Yang, A.; Fan, A.; et al. 2024 · 2024

Closest in time.

Mixture-of-loras: An efficient multitask tuning for large language models

Original

Feng, W.; Hao, C.; Zhang, Y.; Han, Y.; and Wang, H. 2024 · 2024

Closest in time.

Higher Layers Need More LoRA Experts

Original

Gao, C.; Chen, K.; Rao, J.; Sun, B.; Liu, R.; Peng, D.; Zhang, Y.; Guo, X.; Yang, J.; and Subrahmanian, V. 2024 · 2024

Closest in time.

Lora+: Efficient low rank adaptation of large models