Fetching the paper…
Reading the bibliography…
Deep neural networks (DNNs) exhibit a surprising structure in their final layer known as neural collapse (NC), and a growing body of works has currently investigated the propagation of neural collapse to earlier layers of DNNs -- a phenomenon called deep neural collapse (DNC).
Rank-one modification of the symmetric eigenproblem
James R Bunch, Christopher P Nielsen, and Danny C Sorensen · 1978
Earlier work this paper cites.
Gradient-based learning applied to document recognition
Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner · 1998
Earlier work this paper cites.
Learning multiple layers of features from tiny images
Alex Krizhevsky and Geoffrey Hinton · 2009
Earlier work this paper cites.
Gradient descent aligns the layers of deep linear networks
Ziwei Ji and Matus Telgarsky · 2018
Earlier work this paper cites.
The implicit bias of gradient descent on separable data
Daniel Soudry, Elad Hoffer, Mor Shpigel Nacson, Suriya Gunasekar, and Nathan Srebro · 2018
Earlier work this paper cites.
Implicit regularization in deep matrix factorization
Sanjeev Arora, Nadav Cohen, Wei Hu, and Yuping Luo · 2019
Earlier work this paper cites.
Neural collapse with unconstrained features
Dustin G Mixon, Hans Parshall, and Jianzong Pi · 2020
Earlier work this paper cites.
Prevalence of neural collapse during the terminal phase of deep learning training
Vardan Papyan, X. Y. Han, and David L Donoho · 2020
Earlier work this paper cites.
Explicit regularization and implicit bias in deep network classifiers trained with the square loss
Tomaso Poggio and Qianli Liao · 2020
Earlier work this paper cites.
Implicit dynamic regularization in deep networks
Tomaso Poggio and Qianli Liao · 2020
Earlier work this paper cites.
A unified scalable equivalent formulation for Schatten quasi-norms
Fanhua Shang, Yuanyuan Liu, Fanjie Shang, Hongying Liu, Lin Kong, and Licheng Jiao · 2020
Earlier work this paper cites.
Exploring deep neural networks via layer-peeled model: Minority collapse in imbalanced training
Cong Fang, Hangfeng He, Qi Long, and Weijie J Su · 2021
Earlier work this paper cites.
Implicit regularization in tensor factorization
Noam Razin, Asaf Maman, and Nadav Cohen · 2021
Earlier work this paper cites.
A geometric analysis of neural collapse with unconstrained features
Zhihui Zhu, Tianyu Ding, Jinxin Zhou, Xiao Li, Chong You, Jeremias Sulam, and Qing Qu · 2021
Earlier work this paper cites.
Nearest class-center simplification through intermediate layers
Ido Ben-Shaul and Shai Dekel · 2022
Earlier work this paper cites.
Rank diminishing in deep neural networks
Ruili Feng, Kecheng Zheng, Yukun Huang, Deli Zhao, Michael Jordan, and Zheng-Jun Zha · 2022
Earlier work this paper cites.
On the implicit bias towards minimal depth of deep neural networks
Tomer Galanti, Liane Galanti, and Ido Ben-Shaul · 2022
Earlier work this paper cites.
Improved generalization bounds for transfer learning via neural collapse
Tomer Galanti, András György, and Marcus Hutter · 2022
Earlier work this paper cites.
SGD and weight decay provably induce a low-rank bias in neural networks
Tomer Galanti, Zachary S Siegel, Aparna Gupte, and Tomaso Poggio · 2022
Earlier work this paper cites.
Linking neural collapse and l2 normalization with improved out-of-distribution detection in deep neural networks
Jarrod Haas, William Yolland, and Bernhard T Rabus · 2022
Earlier work this paper cites.
Neural collapse under mse loss: Proximity to and dynamics on the central path
X. Y. Han, Vardan Papyan, and David L Donoho · 2022
Earlier work this paper cites.
The low-rank simplicity bias in deep networks
Minyoung Huh, Hossein Mobahi, Richard Zhang, Brian Cheung, Pulkit Agrawal, and Phillip Isola · 2022
Cited alongside, same era.
Limitations of neural collapse for understanding generalization in deep learning
Like Hui, Mikhail Belkin, and Preetum Nakkiran · 2022
Cited alongside, same era.
An unconstrained layer-peeled perspective on neural collapse
Wenlong Ji, Yiping Lu, Yiliang Zhang, Zhun Deng, and Weijie J Su · 2022
Cited alongside, same era.
The asymmetric maximum margin bias of quasi-homogeneous neural networks
Daniel Kunin, Atsushi Yamamura, Chao Ma, and Surya Ganguli · 2022
Cited alongside, same era.
Training invariances and the low-rank phenomenon: beyond linear networks
Thien Le and Stefanie Jegelka · 2022
Cited alongside, same era.
Neural collapse under cross-entropy loss
A neural collapse perspective on feature evolution in graph neural networks
Vignesh Kothapalli, Tom Tirer, and Joan Bruna · 2023
Later among the works it cites.
Principled and efficient transfer learning of deep models via neural collapse
Xiao Li, Sheng Liu, Jinxin Zhou, Xinyu Lu, Carlos Fernandez-Granda, Zhihui Zhu, and Qing Qu · 2023
Later among the works it cites.
No fear of classifier biases: Neural collapse inspired federated learning with synthetic and fixed classifier
Zexi Li, Xinyi Shang, Rui He, Tao Lin, and Chao Wu · 2023
Later among the works it cites.
The tunnel effect: Building data representations in deep neural networks
Wojciech Masarczyk, Mateusz Ostaszewski, Ehsan Imani, Razvan Pascanu, Piotr Miłoś, and Tomasz Trzcinski · 2023
Later among the works it cites.
Towards understanding neural collapse: The effects of batch normalization and weight decay
Leyan Pan and Xinyuan Cao · 2023
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Jianfeng Lu and Stefan Steinerberger · 2022
Cited alongside, same era.
The role of linear layers in nonlinear interpolating networks
Greg Ongie and Rebecca Willett · 2022
Cited alongside, same era.
Imbalance trouble: Revisiting neural-collapse geometry
Christos Thrampoulidis, Ganesh Ramachandra Kini, Vala Vakilian, and Tina Behnia · 2022
Cited alongside, same era.
Extended unconstrained features model for exploring deep neural collapse
Tom Tirer and Joan Bruna · 2022
Cited alongside, same era.
Linear convergence analysis of neural collapse with unconstrained features
Peng Wang, Huikang Liu, Can Yaras, Laura Balzano, and Qing Qu · 2022
Cited alongside, same era.
On the emergence of simplex symmetry in the final and penultimate layers of neural network classifiers
E Weinan and Stephan Wojtowytsch · 2022
Cited alongside, same era.
On the optimization landscape of neural collapse under MSE loss: Global optimality with unconstrained features
Jinxin Zhou, Xiao Li, Tianyu Ding, Chong You, Qing Qu, and Zhihui Zhu · 2022
Cited alongside, same era.
Neural collapse in the intermediate hidden layers of classification neural networks
Liam Parker, Emre Onal, Anton Stengel, and Jake Intrater · 2023
Later among the works it cites.
Feature learning in deep classifiers through intermediate neural collapse
Akshay Rangamani, Marius Lindegaard, Tomer Galanti, and Tomaso Poggio · 2023
Later among the works it cites.
Neural (tangent kernel) collapse
Mariia Seleznova, Dana Weitzner, Raja Giryes, Gitta Kutyniok, and Hung-Hsu Chou · 2023
Later among the works it cites.
Deep neural collapse is provably optimal for the deep unconstrained features model
Peter Súkeník, Marco Mondelli, and Christoph Lampert · 2023
Later among the works it cites.
Implicit regularization towards rank minimization in relu networks
Nadav Timor, Gal Vardi, and Ohad Shamir · 2023
Later among the works it cites.
Perturbation analysis of neural collapse
Tom Tirer, Haoxiang Huang, and Jonathan Niles-Weed · 2023
Later among the works it cites.
Dynamics in deep classifiers trained with the square loss: Normalization, low rank, neural collapse, and generalization bounds
Mengjia Xu, Akshay Rangamani, Qianli Liao, Tomer Galanti, and Tomaso Poggio · 2023
Later among the works it cites.
Bradley T Baker, Barak A Pearlmutter, Robyn Miller, Vince D Calhoun, and Sergey M Plis · 2024
Closest in time.
Average gradient outer product as a mechanism for deep neural collapse
Daniel Beaglehole, Peter Súkeník, Marco Mondelli, and Mikhail Belkin · 2024
Closest in time.
Gradient descent for deep matrix factorization: Dynamics and implicit bias towards low rank
Hung-Hsu Chou, Carsten Gieshoff, Johannes Maly, and Holger Rauhut · 2024
Closest in time.
Neural collapse for cross-entropy class-imbalanced learning with unconstrained ReLU feature model
Hien Dang, Tho Tran, Tan Nguyen, and Nhat Ho · 2024
Closest in time.
Connall Garrod and Jonathan P Keating · 2024
Closest in time.
Implicit bias of SGD in L 2 L_{2} -regularized linear DNNs: One-way jumps from high to low rank
Zihan Wang and Arthur Jacot · 2024
Closest in time.
Which frequencies do CNNs need? Emergent bottleneck structure in feature learning
Yuxiao Wen and Arthur Jacot · 2024
Closest in time.
EPA: neural collapse inspired robust out-of-distribution detector
Jiawei Zhang, Yufan Chen, Cheng Jin, Lei Zhu, and Yuantao Gu · 2024
Closest in time.