Fetching the paper…
Reading the bibliography…
Various parameter-efficient fine-tuning (PEFT) techniques have been proposed to enable computationally efficient fine-tuning while maintaining model performance.
Drop: A reading comprehension benchmark requiring discrete reasoning over paragraphs
Dheeru Dua, Yizhong Wang, Pradeep Dasigi, Gabriel Stanovsky, Sameer Singh, and Matt Gardner. 2019 · 1903
Earlier work this paper cites.
Roberta: A robustly optimized bert pretraining approach
Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019 · 1907
Earlier work this paper cites.
The pascal recognising textual entailment challenge
Ido Dagan, Oren Glickman, and Bernardo Magnini. 2005 · 2005
Earlier work this paper cites.
Adapterfusion: Non-destructive task composition for transfer learning
Jonas Pfeiffer, Aishwarya Kamath, Andreas Rücklé, Kyunghyun Cho, and Iryna Gurevych. 2020 · 2005
Earlier work this paper cites.
Tensor decompositions and applications
Tamara G Kolda and Brett W Bader. 2009 · 2009
Earlier work this paper cites.
Tensor-train decomposition
Ivan V Oseledets. 2011 · 2011
Earlier work this paper cites.
Intrinsic dimensionality explains the effectiveness of language model fine-tuning
Armen Aghajanyan, Luke Zettlemoyer, and Sonal Gupta. 2020 · 2012
Earlier work this paper cites.
Recursive deep models for semantic compositionality over a sentiment treebank
Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D Manning, Andrew Y Ng, and Christopher Potts. 2013 · 2013
Earlier work this paper cites.
Compression of deep convolutional neural networks for fast and low power mobile applications
Yong-Deok Kim, Eunhyeok Park, Sungjoo Yoo, Taelim Choi, Lu Yang, and Dongjun Shin. 2015 · 2015
Earlier work this paper cites.
Speeding-up convolutional neural networks using fine-tuned cp-decomposition
V Lebedev, Y Ganin, M Rakhuba, I Oseledets, and V Lempitsky. 2015 · 2015
Earlier work this paper cites.
Tensorizing neural networks
Alexander Novikov, Dmitrii Podoprikhin, Anton Osokin, and Dmitry P Vetrov. 2015 · 2015
Earlier work this paper cites.
Squad: 100,000+ questions for machine comprehension of text
Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. 2016 · 2016
Earlier work this paper cites.
An overview of multi-task learning in deep neural networks
Sebastian Ruder. 2017 · 2017
Earlier work this paper cites.
A broad-coverage challenge corpus for sentence understanding through inference
Adina Williams, Nikita Nangia, and Samuel R Bowman. 2017 · 2017
Earlier work this paper cites.
Bert: Pre-training of deep bidirectional transformers for language understanding
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018 · 2018
Earlier work this paper cites.
Decoupled weight decay regularization
Ilya Loshchilov and Frank Hutter. 2018 · 2018
Earlier work this paper cites.
Know what you don’t know: Unanswerable questions for squad
Pranav Rajpurkar, Robin Jia, and Percy Liang. 2018 · 2018
Earlier work this paper cites.
Glue: A multi-task benchmark and analysis platform for natural language understanding
Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R Bowman. 2018 · 2018
Earlier work this paper cites.
Neural network acceptability judgments
Alex Warstadt, Amanpreet Singh, and Samuel R Bowman. 2018 · 2018
Earlier work this paper cites.
Parameter-efficient transfer learning for nlp
Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin De Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly. 2019 · 2019
Cited alongside, same era.
Superglue: A stickier benchmark for general-purpose language understanding systems
Alex Wang, Yada Pruksachatkun, Nikita Nangia, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel Bowman. 2019 · 2019
Cited alongside, same era.
Gpt-3: Its nature, scope, limits, and consequences
Luciano Floridi and Massimo Chiriatti. 2020 · 2020
Cited alongside, same era.
Deberta: Decoding-enhanced bert with disentangled attention
Pengcheng He, Xiaodong Liu, Jianfeng Gao, and Weizhu Chen. 2020 · 2020
Cited alongside, same era.
Bayesian tensorized neural networks with automatic rank selection
Cole Hawkins and Zheng Zhang. 2021 · 2021
Cited alongside, same era.
Knowledge distillation circumvents nonlinearity for optical convolutional neural networks
Jinlin Xiang, Shane Colburn, Arka Majumdar, and Eli Shlizerman. 2022 · 2022
Later among the works it cites.
Bitfit: Simple parameter-efficient fine-tuning for transformer-based masked language-models
Elad Ben Zaken, Yoav Goldberg, and Shauli Ravfogel. 2022 · 2022
Later among the works it cites.
Real-time network intrusion detection via decision transformers
Jingdi Chen, Hanhan Zhou, Yongsheng Mei, Gina Adam, Nathaniel D Bastian, and Tian Lan. 2023 · 2023
Later among the works it cites.
Ml-lmcl: Mutual learning and large-margin contrastive learning for improving asr robustness in spoken language understanding
Xuxin Cheng, Bowen Cao, Qichen Ye, Zhihong Zhu, Hongxiang Li, and Yuexian Zou. 2023a · 2023
Later among the works it cites.
Accelerating multiple intent detection and slot filling via targeted knowledge distillation
Xuxin Cheng, Zhihong Zhu, Wanshi Xu, Yaowei Li, Hongxiang Li, and Yuexian Zou. 2023c · 2023
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2021 · 2021
Cited alongside, same era.
Fedpara: Low-rank hadamard product for communication-efficient federated learning
Nam Hyeon-Woo, Moon Ye-Bin, and Tae-Hyun Oh. 2021 · 2021
Cited alongside, same era.
The power of scale for parameter-efficient prompt tuning
Brian Lester, Rami Al-Rfou, and Noah Constant. 2021 · 2021
Cited alongside, same era.
Prefix-tuning: Optimizing continuous prompts for generation
Xiang Lisa Li and Percy Liang. 2021 · 2021
Cited alongside, same era.
Enabling lightweight fine-tuning for pre-trained language model compression based on matrix product operators
Peiyu Liu, Ze-Feng Gao, Wayne Xin Zhao, Zhi-Yuan Xie, Zhong-Yi Lu, and Ji-Rong Wen. 2021 · 2021
Cited alongside, same era.
A survey on multi-task learning
Yu Zhang and Qiang Yang. 2021 · 2021
Cited alongside, same era.
Flashattention: Fast and memory-efficient exact attention with io-awareness
Tri Dao, Dan Fu, Stefano Ermon, Atri Rudra, and Christopher Ré. 2022 · 2022
Cited alongside, same era.
Later among the works it cites.
Parameter-efficient fine-tuning of large-scale pre-trained language models
Ning Ding, Yujia Qin, Guang Yang, Fuchao Wei, Zonghan Yang, Yusheng Su, Shengding Hu, Yulin Chen, Chi-Min Chan, Weize Chen, et al. 2023 · 2023
Later among the works it cites.
Msq-biobert: Ambiguity resolution to enhance biobert medical question-answering
Muzhe Guo, Muhao Guo, Edward T Dougherty, and Fang Jin. 2023a · 2023
Later among the works it cites.
Llm-adapters: An adapter family for parameter-efficient fine-tuning of large language models
Zhiqiang Hu, Yihuai Lan, Lei Wang, Wanyu Xu, Ee-Peng Lim, Roy Ka-Wei Lee, Lidong Bing, and Soujanya Poria. 2023 · 2023
Later among the works it cites.
Fact: Factor-tuning for lightweight adaptation on vision transformer
Shibo Jie and Zhi-Hong Deng. 2023 · 2023
Later among the works it cites.
Prefix propagation: Parameter-efficient tuning for long sequences
Jonathan Li, Will Aitken, Rohan Bhambhoria, and Xiaodan Zhu. 2023 · 2023
Later among the works it cites.
Parameter-efficient orthogonal finetuning via butterfly factorization
Weiyang Liu, Zeju Qiu, Yao Feng, Yuliang Xiu, Yuxuan Xue, Longhui Yu, Haiwen Feng, Zhen Liu, Juyeon Heo, Songyou Peng, et al. 2023 · 2023
Later among the works it cites.
A transfer learning framework for proactive ramp metering performance assessment
Xiaobo Ma, Adrian Cottam, Mohammad Razaur Rahman Shaon, and Yao-Jan Wu. 2023 · 2023
Later among the works it cites.
Fine-tuning language models with just forward passes
Sadhika Malladi, Tianyu Gao, Eshaan Nichani, Alex Damian, Jason D Lee, Danqi Chen, and Sanjeev Arora. 2023 · 2023
Later among the works it cites.
Llama: Open and efficient foundation language models
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. 2023 · 2023
Later among the works it cites.
Msd: Mixing signed digit representations for hardware-efficient dnn acceleration on fpga with heterogeneous resources
Jiajun Wu, Jiajun Zhou, Yizhao Gao, Yuhao Ding, Ngai Wong, and Hayden Kwok-Hay So. 2023 · 2023
Later among the works it cites.
Dybit: Dynamic bit-precision numbers for efficient quantized neural network inference
Jiajun Zhou, Jiajun Wu, Yizhao Gao, Yuhao Ding, Chaofan Tao, Boyu Li, Fengbin Tu, Kwang-Ting Cheng, Hayden Kwok-Hay So, and Ngai Wong. 2023 · 2023
Later among the works it cites.
Data-driven transfer learning framework for estimating on-ramp and off-ramp traffic flows
Xiaobo Ma, Abolfazl Karimpour, and Yao-Jan Wu. 2024 · 2024
Closest in time.
Projection-optimal monotonic value function factorization in multi-agent reinforcement learning
Yongsheng Mei, Hanhan Zhou, and Tian Lan. 2024 · 2024
Closest in time.