Fetching the paper…
Reading the bibliography…
Fine-tuning large pre-trained language models for downstream tasks remains a critical challenge in natural language processing.
Language models are few-shot learners
Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J. D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · 1901
Earlier work this paper cites.
Fine-tuning language models from human preferences
Ziegler, D. M., Stiennon, N., Wu, J., Brown, T. B., Radford, A., Amodei, D., Christiano, P., and Irving, G. (2019) · 1909
Earlier work this paper cites.
Fine-tuning pretrained language models: Weight initializations, data orders, and early stopping
Dodge, J., Ilharco, G., Schwartz, R., Farhadi, A., Hajishirzi, H., and Smith, N. (2020) · 2002
Earlier work this paper cites.
Why does unsupervised pre-training help deep learning?
Erhan, D., Courville, A., Bengio, Y., and Vincent, P. (2010) · 2010
Earlier work this paper cites.
Learning multiple visual domains with residual adapters
Rebuffi, S.-A., Bilen, H., and Vedaldi, A. (2017) · 2017
Earlier work this paper cites.
Bert: Pre-training of deep bidirectional transformers for language understanding
Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K. (2018) · 2018
Earlier work this paper cites.
Universal language model fine-tuning for text classification
Howard, J. and Ruder, S. (2018) · 2018
Earlier work this paper cites.
Glue: A multi-task benchmark and analysis platform for natural language understanding
Wang, A., Singh, A., Michael, J., Hill, F., Levy, O., and Bowman, S. R. (2018) · 2018
Earlier work this paper cites.
Reconciling modern machine-learning practice and the classical bias–variance trade-off
Belkin, M., Hsu, D., Ma, S., and Mandal, S. (2019) · 2019
Cited alongside, same era.
Parameter-efficient transfer learning for nlp
Houlsby, N., Giurgiu, A., Jastrzebski, S., Morrone, B., De Laroussilhe, Q., Gesmundo, A., Attariyan, M., and Gelly, S. (2019) · 2019
Cited alongside, same era.
Biobert: A pre-trained biomedical language representation model for biomedical text mining
Lee, J., Yoon, W., Kim, S., Kim, D., Kim, S., So, C. H., and Kang, J. (2019) · 2019
Cited alongside, same era.
Exploring the limits of transfer learning with a unified text-to-text transformer
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., and Liu, P. J. (2019) · 2019
Cited alongside, same era.
How fine can fine-tuning be? learning efficient language models
Radiya-Dixit, E. and Wang, X. (2020) · 2020
Cited alongside, same era.
Convfit: Conversational fine-tuning of pretrained language models
Vulić, I., Su, P.-H., Coope, S., Gerz, D., Budzianowski, P., Casanueva, I., Mrkšić, N., and Wen, T.-H. (2021) · 2021
Later among the works it cites.
Differentially private fine-tuning of language models
Yu, D., Naik, S., Backurs, A., Gopi, S., Inan, H. A., Kamath, G., Kulkarni, J., Lee, Y. T., Manoel, A., Wutschitz, L., et al. (2021) · 2021
Later among the works it cites.
Bitfit: Simple parameter-efficient fine-tuning for transformer-based masked language-models
Zaken, E. B., Ravfogel, S., and Goldberg, Y. (2021) · 2021
Later among the works it cites.
Adamix: Mixture-of-adaptations for parameter-efficient model tuning
Wang, Y., Agarwal, S., Mukherjee, S., Liu, X., Gao, J., Awadallah, A. H., and Gao, J. (2022) · 2022
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
He, R., Liu, L., Ye, H., Tan, Q., Ding, B., Cheng, L., Low, J.-W., Bing, L., and Si, L. (2021) · 2021
Cited alongside, same era.
Learning transferable visual models from natural language supervision
Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., and Sutskever, I. (2021) · 2021
Cited alongside, same era.
Bubeck, S., Chandrasekaran, V., Eldan, R., Gehrke, J., Horvitz, E., Kamar, E., Lee, P., Lee, Y. T., Li, Y., Lundberg, S., et al. (2023) · 2023
Later among the works it cites.
Training data eigenvector dynamics in the eigenpro implementation of the neural tangent kernel and recursive feature machines
Gorlla, C. (2023) · 2023
Later among the works it cites.
Peng, B., Li, C., He, P., Galley, M., and Gao, J. (2023) · 2023
Later among the works it cites.