Fetching the paper…
Reading the bibliography…
We study the computational limits of Low-Rank Adaptation (LoRA) for finetuning transformer-based models using fine-grained complexity theory.
On the complexity of k-sat
Russell Impagliazzo and Ramamohan Paturi · 2001
Earlier work this paper cites.
The complexity of unique k-sat: An isolation lemma for k-cnfs
Chris Calabro, Russell Impagliazzo, and Ramamohan Paturi · 2009
Earlier work this paper cites.
Fast approximation algorithms for the diameter and radius of sparse graphs
Liam Roditty and Virginia Vassilevska Williams · 2013
Earlier work this paper cites.
Lecture 24: Hardness assumptions
A Theorist’s Toolkit · 2013
Earlier work this paper cites.
Finding paths of length k in o ∗ ( 2 k ) o^{*}(2^{k}) time
Ryan Williams · 2013
Earlier work this paper cites.
Consequences of faster alignment of sequences
Amir Abboud, Virginia Vassilevska Williams, and Oren Weimann · 2014
Earlier work this paper cites.
Why walking the dog takes time: Frechet distance has no strongly subquadratic algorithms unless seth fails
Karl Bringmann · 2014
Earlier work this paper cites.
Which regular expression patterns are hard to match?
Arturs Backurs and Piotr Indyk · 2016
Earlier work this paper cites.
Approximability of the discrete fréchet distance
Karl Bringmann and Wolfgang Mulzer · 2016
Earlier work this paper cites.
Fine-grained analysis of problems on curves
Kevin Buchin, Maike Buchin, Maximilian Konzack, Wolfgang Mulzer, and André Schulz · 2016
Earlier work this paper cites.
On problems as hard as cnf-sat
Marek Cygan, Holger Dell, Daniel Lokshtanov, Dániel Marx, Jesper Nederlof, Yoshio Okamoto, Ramamohan Paturi, Saket Saurabh, and Magnus Wahlström · 2016
Earlier work this paper cites.
On the fine-grained complexity of empirical risk minimization: Kernel methods and neural networks
Arturs Backurs, Piotr Indyk, and Ludwig Schmidt · 2017
Earlier work this paper cites.
A dichotomy for regular expression membership testing
Karl Bringmann, Allan Grønlund, and Kasper Green Larsen · 2017
Earlier work this paper cites.
Attention is all you need
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin · 2017
Earlier work this paper cites.
Subtree isomorphism revisited
Amir Abboud, Arturs Backurs, Thomas Dueholm Hansen, Virginia Vassilevska Williams, and Or Zamir · 2018
Earlier work this paper cites.
Multivariate fine-grained complexity of longest common subsequence
Karl Bringman and Marvin Künnemann · 2018
Earlier work this paper cites.
On the hardness of approximate and exact (bichromatic) maximum inner product
Lijie Chen · 2018
Earlier work this paper cites.
Sketching for kronecker product regression and p-splines
Huaian Diao, Zhao Song, Wen Sun, and David Woodruff · 2018
Earlier work this paper cites.
Completeness for first-order properties on sparse structures with algorithmic applications
Jiawei Gao, Russell Impagliazzo, Antonina Kolokolova, and Ryan Williams · 2018
Earlier work this paper cites.
Conditional lower bounds for all-pairs max-flow
Robert Krauthgamer and Ohad Trabelsi · 2018
Earlier work this paper cites.
Hardness of approximate nearest neighbor search
Aviad Rubinstein · 2018
Earlier work this paper cites.
On some fine-grained questions in algorithms and complexity
Virginia Vassilevska Williams · 2018
Earlier work this paper cites.
An equivalence class for orthogonal vectors
Lijie Chen and Ryan Williams · 2019
Earlier work this paper cites.
Optimal sketching for kronecker product regression and low rank approximation
Huaian Diao, Rajesh Jayaram, Zhao Song, Wen Sun, and David Woodruff · 2019
Cited alongside, same era.
Learning deep transformer models for machine translation
Qiang Wang, Bei Li, Tong Xiao, Jingbo Zhu, Changliang Li, Derek F Wong, and Lidia S Chao · 2019
Cited alongside, same era.
Algorithms and hardness for linear algebra on geometric graphs
Josh Alman, Timothy Chu, Aaron Schild, and Zhao Song · 2020
Cited alongside, same era.
Language models are few-shot learners
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al · 2020
Cited alongside, same era.
Gpt-3: Its nature, scope, limits, and consequences
Luciano Floridi and Massimo Chiriatti · 2020
Cited alongside, same era.
Foundation models for generalist medical artificial intelligence
Michael Moor, Oishi Banerjee, Zahra Shakeri Hossein Abad, Harlan M Krumholz, Jure Leskovec, Eric J Topol, and Pranav Rajpurkar · 2023
Later among the works it cites.
Large language models encode clinical knowledge
Karan Singhal, Shekoofeh Azizi, Tao Tu, S Sara Mahdavi, Jason Wei, Hyung Won Chung, Nathan Scales, Ajay Tanwani, Heather Cole-Lewis, Stephen Pfohl, et al · 2023
Later among the works it cites.
Large language models in medicine
Arun James Thirunavukarasu, Darren Shu Jeng Ting, Kabilan Elangovan, Laura Gutierrez, Ting Fang Tan, and Daniel Shu Wei Ting · 2023
Later among the works it cites.
Bloomberggpt: A large language model for finance
Shijie Wu, Ozan Irsoy, Steven Lu, Vadim Dabravolski, Mark Dredze, Sebastian Gehrmann, Prabhanjan Kambadur, David Rosenberg, and Gideon Mann · 2023
Later among the works it cites.
Fingpt: Open-source financial large language models
Hongyang Yang, Xiao-Yang Liu, and Christina Dan Wang · 2023
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Jacob Kahn, Morgane Riviere, Weiyi Zheng, Evgeny Kharitonov, Qiantong Xu, Pierre-Emmanuel Mazaré, Julien Karadayi, Vitaliy Liptchinsky, Ronan Collobert, Christian Fuegen, et al · 2020
Cited alongside, same era.
On closest pair in euclidean metric: Monochromatic is as hard as bichromatic
CS Karthik and Pasin Manurangsi · 2020
Cited alongside, same era.
Dylora: Towards energy efficient dynamic lora transmission control
Yinghui Li, Jing Yang, and Jiliang Wang · 2020
Cited alongside, same era.
On layer normalization in the transformer architecture
Ruibin Xiong, Yunchang Yang, Di He, Kai Zheng, Shuxin Zheng, Chen Xing, Huishuai Zhang, Yanyan Lan, Liwei Wang, and Tieyan Liu · 2020
Cited alongside, same era.
On the opportunities and risks of foundation models
Rishi Bommasani, Drew A Hudson, Ehsan Adeli, Russ Altman, Simran Arora, Sydney von Arx, Michael S Bernstein, Jeannette Bohg, Antoine Bosselut, Emma Brunskill, et al · 2021
Cited alongside, same era.
Lora: Low-rank adaptation of large language models
Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen · 2021
Cited alongside, same era.
Dnabert: pre-trained bidirectional encoder representations from transformers model for dna-language in genome
Yanrong Ji, Zhihan Zhou, Han Liu, and Ramana V Davuluri · 2021
Cited alongside, same era.
Later among the works it cites.
Adaptive budget allocation for parameter-efficient fine-tuning
Qingru Zhang, Minshuo Chen, Alexander Bukharin, Pengcheng He, Yu Cheng, Weizhu Chen, and Tuo Zhao · 2023
Later among the works it cites.
Dnabert-2: Efficient foundation model and benchmark for multi-species genome
Zhihan Zhou, Yanrong Ji, Weijian Li, Pratik Dutta, Ramana Davuluri, and Han Liu · 2023
Later among the works it cites.
Fundamental limitations on subquadratic alternatives to transformers
Josh Alman and Hantao Yu · 2024
Closest in time.
Qlora: Efficient finetuning of quantized llms
Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer · 2024
Closest in time.
LQ-loRA: Low-rank plus quantized matrix decomposition for efficient language model finetuning
Han Guo, Philip Greengard, Eric Xing, and Yoon Kim · 2024
Closest in time.
Lora+: Efficient low rank adaptation of large models
Soufiane Hayou, Nikhil Ghosh, and Bin Yu · 2024
Closest in time.
Loftq: LoRA-fine-tuning-aware quantization for large language models
Yixiao Li, Yifan Yu, Chen Liang, Nikos Karampatziakis, Pengcheng He, Weizhu Chen, and Tuo Zhao · 2024
Closest in time.
Dora: Weight-decomposed low-rank adaptation
Shih-Yang Liu, Chien-Yi Wang, Hongxu Yin, Pavlo Molchanov, Yu-Chiang Frank Wang, Kwang-Ting Cheng, and Min-Hung Chen · 2024
Closest in time.
Voxtlm: Unified decoder-only models for consolidating speech recognition, synthesis and speech, text continuation tasks
Soumi Maiti, Yifan Peng, Shukjae Choi, Jee-weon Jung, Xuankai Chang, and Shinji Watanabe · 2024
Closest in time.
Hyenadna: Long-range genomic sequence modeling at single nucleotide resolution
Eric Nguyen, Michael Poli, Marjan Faizi, Armin Thomas, Michael Wornow, Callum Birch-Sykes, Stefano Massaroli, Aman Patel, Clayton Rabideau, Yoshua Bengio, et al · 2024
Closest in time.
Lisa: Layerwise importance sampling for memory-efficient large language model fine-tuning
Rui Pan, Xiang Liu, Shizhe Diao, Renjie Pi, Jipeng Zhang, Chi Han, and Tong Zhang · 2024
Closest in time.
Massive activations in large language models
Mingjie Sun, Xinlei Chen, J Zico Kolter, and Zhuang Liu · 2024
Closest in time.
The expressive power of low-rank adaptation
Yuchen Zeng and Kangwook Lee · 2024
Closest in time.
Llamafactory: Unified efficient fine-tuning of 100+ language models
Yaowei Zheng, Richong Zhang, Junhao Zhang, Yanhan Ye, and Zheyan Luo · 2024
Closest in time.
Dnabert-s: Learning species-aware dna embedding with genome foundation models
Zhihan Zhou, Winmin Wu, Harrison Ho, Jiayi Wang, Lizhen Shi, Ramana V Davuluri, Zhong Wang, and Han Liu · 2024
Closest in time.
Provably optimal memory capacity for modern hopfield models: Transformer-compatible dense associative memories as spherical codes
Jerry Yao-Chieh Hu, Dennis Wu, and Han Liu · 2025
Closest in time.
A survey on lora of large language models
Yuren Mao, Yuhang Ge, Yijiang Fan, Wenyi Xu, Yu Mi, Zhonghao Hu, and Yunjun Gao · 2025
Closest in time.
Genomeocean: An efficient genome foundation model trained on large-scale metagenomic assemblies
Zhihan Zhou, Robert Riley, Satria Kautsar, Weimin Wu, Rob Egan, Steven Hofmeyr, Shira Goldhaber-Gordon, Mutian Yu, Harrison Ho, Fengchen Liu, et al · 2025
Closest in time.