Improving Small Language Models on PubMedQA via Generative Data Augmentation

Few-shot parameter-efficient fine-tuning is better and cheaper than in-context learning,

H. Liu, D. Tam, M. Muqeeth, J. Mohta, T. Huang, M. Bansal, C. A. Raffel, · 1965

Earlier work this paper cites.

Decoupled weight decay regularization,

Original

I. Loshchilov, F. Hutter, · 2017

Earlier work this paper cites.

Attention is all you need,

A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, I. Polosukhin, · 2017

Earlier work this paper cites.

Deep gradient compression: Reducing the communication bandwidth for distributed training,

Original

Y. Lin, S. Han, H. Mao, Y. Wang, W. J. Dally, · 2017

Earlier work this paper cites.

Interactive supercomputing on 40,000 cores for machine learning and data analysis,

A. Reuther, J. Kepner, C. Byun, S. Samsi, W. Arcand, D. Bestor, B. Bergeron, V. Gadepally, M. Houle, M. Hubbell, M. Jones, A. Klein, L. Milechin, J. Mullen, A. Prout, A. Rosa, C. Yee, P. Michaleas, · 2018

Earlier work this paper cites.

Energy and policy considerations for deep learning in nlp,

Original

E. Strubell, A. Ganesh, A. McCallum, · 2019

Earlier work this paper cites.

Pubmedqa: A dataset for biomedical research question answering,

Original

Q. Jin, B. Dhingra, Z. Liu, W. W. Cohen, X. Lu, · 2019

Earlier work this paper cites.

Inexpensive domain adaptation of pretrained language models: Case studies on biomedical ner and covid-19 qa,

Original

N. Poerner, U. Waltinger, H. Schütze, · 2020

Earlier work this paper cites.

Squeezebert: What can computer vision teach nlp about efficient neural networks?,

Original

F. N. Iandola, A. E. Shaw, R. Krishna, K. W. Keutzer, · 2020

Earlier work this paper cites.

Program synthesis with large language models,

Original

J. Austin, A. Odena, M. Nye, M. Bosma, H. Michalewski, D. Dohan, E. Jiang, C. Cai, M. Terry, Q. Le, et al., · 2021

Cited alongside, same era.

Evaluating large language models trained on code,

Original

M. Chen, J. Tworek, H. Jun, Q. Yuan, H. P. d. O. Pinto, J. Kaplan, H. Edwards, Y. Burda, N. Joseph, G. Brockman, et al., · 2021

Cited alongside, same era.

Domain-specific language model pretraining for biomedical natural language processing,

Y. Gu, R. Tinn, H. Cheng, M. Lucas, N. Usuyama, X. Liu, T. Naumann, J. Gao, H. Poon, · 2021

Cited alongside, same era.

Few-shot question answering by pretraining span selection,

Original

O. Ram, Y. Kirstain, J. Berant, A. Globerson, O. Levy, · 2021

Cited alongside, same era.

Prefix-tuning: Optimizing continuous prompts for generation,

Towards parameter-efficient automation of data wrangling tasks with prefix-tuning,

D. Vos, T. Döhmen, S. Schelter, · 2022

Later among the works it cites.

Biogpt: generative pre-trained transformer for biomedical text generation and mining,

R. Luo, L. Sun, Y. Xia, T. Qin, S. Zhang, H. Poon, T.-Y. Liu, · 2022

Later among the works it cites.

Z. Guo, P. Wang, Y. Wang, S. Yu, Dr. LLaMA: Improving small language models in domain-specific qa via generative data augmentation, 2023. URL: https://github.com/zguo0525/Dr.LLaMA

2023

Closest in time.

A survey of large language models,

Original

W. X. Zhao, K. Zhou, J. Li, T. Tang, X. Wang, Y. Hou, Y. Min, B. Zhang, J. Zhang, Z. Dong, et al., · 2023

Closest in time.

A short survey of viewing large language models in legal aspect,

Original

Z. Sun, · 2023