Fetching the paper…
Reading the bibliography…
Large language models (LLMs) can acquire new knowledge through fine-tuning, but this process exhibits a puzzling duality: models can generalize remarkably from new facts, yet are also prone to hallucinating incorrect information.
Guaranteed minimum-rank solutions of linear matrix equations via nuclear norm minimization
Benjamin Recht, Maryam Fazel, and Pablo A Parrilo · 2010
Earlier work this paper cites.
Adam: A method for stochastic optimization
Diederik P Kingma · 2014
Earlier work this paper cites.
CVXPY: A Python-embedded modeling language for convex optimization
Steven Diamond and Stephen Boyd · 2016
Earlier work this paper cites.
Implicit regularization in matrix factorization
Suriya Gunasekar, Blake E Woodworth, Srinadh Bhojanapalli, Behnam Neyshabur, and Nati Srebro · 2017
Earlier work this paper cites.
Algorithmic regularization in over-parameterized matrix sensing and neural networks with quadratic activations
Yuanzhi Li, Tengyu Ma, and Hongyang Zhang · 2018
Earlier work this paper cites.
The implicit bias of gradient descent on separable data
Daniel Soudry, Elad Hoffer, Mor Shpigel Nacson, Suriya Gunasekar, and Nathan Srebro · 2018
Earlier work this paper cites.
Implicit regularization in deep matrix factorization
Sanjeev Arora, Nadav Cohen, Wei Hu, and Yuping Luo · 2019
Earlier work this paper cites.
The implicit bias of gradient descent on nonseparable data
Ziwei Ji and Matus Telgarsky · 2019
Earlier work this paper cites.
Gradient descent maximizes the margin of homogeneous neural networks
Kaifeng Lyu and Jian Li · 2019
Earlier work this paper cites.
Zhiyuan Li, Yuping Luo, and Kaifeng Lyu · 2020
Earlier work this paper cites.
Implicit regularization in deep learning may not be explainable by norms
Noam Razin and Nadav Cohen · 2020
Earlier work this paper cites.
Small random initialization is akin to spectral learning: Optimization and generalization guarantees for overparameterized low-rank matrix reconstruction
Dominik Stöger and Mahdi Soltanolkotabi · 2021
Earlier work this paper cites.
Learning to reason with neural networks: Generalization, unseen data and boolean measures
Emmanuel Abbe, Samy Bengio, Elisabetta Cornacchia, Jon Kleinberg, Aryo Lotfi, Maithra Raghu, and Chiyuan Zhang · 2022
Earlier work this paper cites.
Vision transformers provably learn spatial structure
Samy Jelassi, Michael Sander, and Yuanzhi Li · 2022
Earlier work this paper cites.
Alex Mallen, Akari Asai, Victor Zhong, Rajarshi Das, Daniel Khashabi, and Hannaneh Hajishirzi · 2022
Earlier work this paper cites.
On margin maximization in linear and relu networks
Gal Vardi, Ohad Shamir, and Nati Srebro · 2022
Cited alongside, same era.
Physics of language models: Part 3.2, knowledge manipulation
Zeyuan Allen-Zhu and Yuanzhi Li · 2023
Cited alongside, same era.
Birth of a transformer: A memory viewpoint
Alberto Bietti, Vivien Cabannes, Diane Bouchacourt, Herve Jegou, and Leon Bottou · 2023
Cited alongside, same era.
Transformers learn through gradual rank increase
Enric Boix-Adsera, Etai Littwin, Emmanuel Abbe, Samy Bengio, and Joshua Susskind · 2023
Cited alongside, same era.
What can a single attention layer learn? a study through the random features lens
Hengyu Fu, Tianyu Guo, Yu Bai, and Song Mei · 2023
Cited alongside, same era.
Uniqueness in nuclear norm minimization: Flatness of the nuclear norm sphere and simultaneous polarization
In-context convergence of transformers
Yu Huang, Yuan Cheng, and Yingbin Liang · 2024
Later among the works it cites.
From self-attention to markov models: Unveiling the dynamics of generative transformers
M Emrullah Ildiz, Yixiao Huang, Yingcong Li, Ankit Singh Rawat, and Samet Oymak · 2024
Later among the works it cites.
Unfamiliar finetuning examples control how language models hallucinate
Katie Kang, Eric Wallace, Claire Tomlin, Aviral Kumar, and Sergey Levine · 2024
Later among the works it cites.
Mechanics of next token prediction with self-attention
Yingcong Li, Yixiao Huang, Muhammed E Ildiz, Ankit Singh Rawat, and Samet Oymak · 2024
Later among the works it cites.
Implicit regularization of gradient flow on one-layer softmax attention
Heejune Sheen, Siyu Chen, Tianhao Wang, and Harrison H Zhou · 2024
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Tim Hoheisel and Elliot Paquette · 2023
Cited alongside, same era.
Out-of-context meta-learning in large language models
Dmitrii Krasheninnikov, Egor Krasheninnikov, and David Krueger · 2023
Cited alongside, same era.
Arvind Mahankali, Tatsunori B Hashimoto, and Tengyu Ma · 2023
Cited alongside, same era.
Generalization on the unseen, logic reasoning and degree curriculum
Emmanuel Abbe, Samy Bengio, Aryo Lotfi, and Kevin Rizk · 2024
Cited alongside, same era.
Hopping too late: Exploring the limitations of large language models on multi-hop queries
Eden Biran, Daniela Gottesman, Sohee Yang, Mor Geva, and Amir Globerson · 2024
Cited alongside, same era.
Evaluating the ripple effects of knowledge editing in language models
Roi Cohen, Eden Biran, Ori Yoran, Amir Globerson, and Mor Geva · 2024
Cited alongside, same era.
Extractive structures learned in pretraining enable generalization on finetuned facts
Jiahai Feng, Stuart Russell, and Jacob Steinhardt · 2024
Cited alongside, same era.
Later among the works it cites.
Implicit bias and fast convergence rates for self-attention
Bhavya Vasudeva, Puneesh Deora, and Christos Thrampoulidis · 2024
Later among the works it cites.
Kaiyue Wen, Huaqing Zhang, Hongzhou Lin, and Jingzhao Zhang · 2024
Later among the works it cites.
Do large language models latently perform multi-hop reasoning?
Sohee Yang, Elena Gribovskaya, Nora Kassner, Mor Geva, and Sebastian Riedel · 2024
Later among the works it cites.
Trained transformers learn linear models in-context
Ruiqi Zhang, Spencer Frei, and Peter L Bartlett · 2024
Later among the works it cites.
Towards a theoretical understanding of the’reversal curse’via training dynamics
Hanlin Zhu, Baihe Huang, Shaolun Zhang, Michael Jordan, Jiantao Jiao, Yuandong Tian, and Stuart J Russell · 2024
Later among the works it cites.
How do llms perform two-hop reasoning in context?
Tianyu Guo, Hanlin Zhu, Ruiqi Zhang, Jiantao Jiao, Song Mei, Michael I Jordan, and Stuart Russell · 2025
Closest in time.
Linear correlation in lm’s compositional generalization and hallucination
Letian Peng, Chenyang An, Shibo Hao, Chengyu Dong, and Jingbo Shang · 2025
Closest in time.
How new data permeates llm knowledge and how to dilute it
Chen Sun, Renat Aksitov, Andrey Zhmoginov, Nolan Andrew Miller, Max Vladymyrov, Ulrich Rueckert, Been Kim, and Mark Sandler · 2025
Closest in time.
Training dynamics of in-context learning in linear attention
Yedi Zhang, Aaditya K Singh, Peter E Latham, and Andrew Saxe · 2025
Closest in time.