TagRouter: Learning Route to LLMs through Tags for Open-Domain Text Generation Tasks

Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter

Original

Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. 2020 · 1910

Earlier work this paper cites.

Routing to the expert: Efficient reward-guided ensemble of large language models

Keming Lu, Hongyi Yuan, Runji Lin, Junyang Lin, Zheng Yuan, Chang Zhou, and Jingren Zhou. 2024b · 1974

Earlier work this paper cites.

The singular value decomposition: Its computation and some applications

V. Klema and A. Laub. 1980 · 1980

Earlier work this paper cites.

Adaptive mixtures of local experts

Robert A Jacobs, Michael I Jordan, Steven J Nowlan, and Geoffrey E Hinton. 1991 · 1991

Earlier work this paper cites.

Scaling laws for neural language models

Original

Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B. Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. 2020 · 2001

Earlier work this paper cites.

BERT: Pre-training of deep bidirectional transformers for language understanding

Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019 · 2019

Earlier work this paper cites.

Decoupled weight decay regularization

Original

Ilya Loshchilov and Frank Hutter. 2019 · 2019

Earlier work this paper cites.

Retrieval-augmented generation for knowledge-intensive nlp tasks

Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, et al. 2020 · 2020

Earlier work this paper cites.

Measuring massive multitask language understanding

Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2021 · 2021

Earlier work this paper cites.

Lora: Low-rank adaptation of large language models

Original

Edward J Hu, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al. 2021 · 2021

Earlier work this paper cites.

Minilmv2: Multi-head self-attention relation distillation for compressing pretrained transformers

Original

Wenhui Wang, Hangbo Bao, Shaohan Huang, Li Dong, and Furu Wei. 2021b · 2021

Earlier work this paper cites.

Chain-of-thought prompting elicits reasoning in large language models

Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. 2022 · 2022

Earlier work this paper cites.

Automix: Automatically mixing language models

Original

Pranjal Aggarwal, Aman Madaan, Ankit Anand, Srividya Pranavi Potharaju, Swaroop Mishra, Pei Zhou, Aditya Gupta, Dheeraj Rajagopal, Karthik Kappaganthu, Yiming Yang, et al. 2023 · 2023

Earlier work this paper cites.

Frugalgpt: How to use large language models while reducing cost and improving performance

Original

Lingjiao Chen, Matei Zaharia, and James Zou. 2023 · 2023

Earlier work this paper cites.

Free dolly: Introducing the world’s first truly open instruction-tuned llm

Mike Conover, Matt Hayes, Ankit Mathur, Jianwei Xie, Jun Wan, Sam Shah, Ali Ghodsi, Patrick Wendell, Matei Zaharia, and Reynold Xin. 2023 · 2023

Earlier work this paper cites.

Trapping llm hallucinations using tagged context prompts

Original

Philip Feldman, James R Foulds, and Shimei Pan. 2023 · 2023

Earlier work this paper cites.

Tryage: Real-time, intelligent routing of user prompts to large language model

Original

Surya Narayanan Hari and Matt Thomson. 2023 · 2023

Earlier work this paper cites.

Llm-blender: Ensembling large language models with pairwise ranking and generative fusion

Dongfu Jiang, Xiang Ren, and Bill Yuchen Lin. 2023 · 2023

Cited alongside, same era.

Fast inference from transformers via speculative decoding

Original

Yaniv Leviathan, Matan Kalman, and Yossi Matias. 2023 · 2023

Cited alongside, same era.

Cmmlu: Measuring massive multitask language understanding in chinese

Original

Haonan Li, Yixuan Zhang, Fajri Koto, Yifei Yang, Hai Zhao, Yeyun Gong, Nan Duan, and Timothy Baldwin. 2023 · 2023

Cited alongside, same era.

A comprehensive survey of few-shot learning: Evolution, applications, challenges, and opportunities

Yisheng Song, Ting Wang, Puyu Cai, Subrota K Mondal, and Jyoti Prakash Sahoo. 2023 · 2023

Cited alongside, same era.

Self-instruct: Aligning language models with self-generated instructions

Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A Smith, Daniel Khashabi, and Hannaneh Hajishirzi. 2023 · 2023

Routellm: Learning to route llms with preference data

Original

Isaac Ong, Amjad Almahairi, Vincent Wu, Wei-Lin Chiang, Tianhao Wu, Joseph E Gonzalez, M Waleed Kadous, and Ion Stoica. 2024 · 2024

Later among the works it cites.

Gpt-4 technical report

Original

OpenAI. 2024 · 2024

Later among the works it cites.

Gorilla: Large language model connected with massive apis

Shishir G Patil, Tianjun Zhang, Xin Wang, and Joseph E Gonzalez. 2024 · 2024

Later among the works it cites.

Tell me more! towards implicit user intention understanding of language model driven agents

Cheng Qian, Bingxiang He, Zhong Zhuang, Jia Deng, Yujia Qin, Xin Cong, Zhong Zhang, Jie Zhou, Yankai Lin, Zhiyuan Liu, and Maosong Sun. 2024 · 2024

Later among the works it cites.

A review on large language models: Architectures, applications, taxonomies, open issues and challenges

Mohaimenul Azam Khan Raiaan, Md. Saddam Hossain Mukta, Kaniz Fatema, Nur Mohammad Fahad, Sadman Sakib, Most Marufatul Jannat Mim, Jubaer Ahmad, Mohammed Eunus Ali, and Sami Azam. 2024 · 2024