Fetching the paper…
Reading the bibliography…
Recent results in the literature suggest that the penultimate (second-to-last) layer representations of neural networks that are trained for classification exhibit a clustering property called neural collapse (NC).
Alphastar: An evolutionary computation perspective, 2019
Kai Arulkumaran, Antoine Cully, and Julian Togelius · 1902
Earlier work this paper cites.
Statistical Learning Theory
Vladimir N. Vapnik · 1998
Earlier work this paper cites.
Rademacher and gaussian complexities: Risk bounds and structural results
Peter L. Bartlett and Shahar Mendelson · 2001
Earlier work this paper cites.
Rademacher and gaussian complexities: Risk bounds and structural results
Peter L. Bartlett and Shahar Mendelson · 2002
Earlier work this paper cites.
Kernel analysis of deep networks
Grégoire Montavon, Mikio L. Braun, and Klaus-Robert Müller · 2011
Earlier work this paper cites.
Foundations of Machine Learning
Mehryar Mohri, Afshin Rostamizadeh, and Ameet Talwalkar · 2012
Earlier work this paper cites.
Understanding Machine Learning - From Theory to Algorithms
Shai Shalev-Shwartz and Shai Ben-David · 2014
Earlier work this paper cites.
Deepface: Closing the gap to human-level performance in face verification
Yaniv Taigman, Ming Yang, Marc’Aurelio Ranzato, and Lior Wolf · 2014
Earlier work this paper cites.
Delving deep into rectifiers: Surpassing human-level performance on imagenet
Kaiming He et al · 2015
Earlier work this paper cites.
Norm-based capacity control in neural networks
Behnam Neyshabur, Ryota Tomioka, and Nathan Srebro · 2015
Earlier work this paper cites.
Deep residual learning for image recognition
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun · 2016
Earlier work this paper cites.
Mastering the game of Go with deep neural networks and tree search
David Silver, Aja Huang, Chris J. Maddison, Arthur Guez, Laurent Sifre, George van den Driessche, Julian Schrittwieser, Ioannis Antonoglou, Veda Panneershelvam, Marc Lanctot, Sander Dieleman, Dominik Grewe, John Nham, Nal Kalchbrenner, Ilya Sutskever, Timothy Lillicrap, Madeleine Leach, Koray Kavukcuoglu, Thore Graepel, and Demis Hassabis · 2016
Earlier work this paper cites.
Understanding intermediate layers using linear classifier probes
Guillaume Alain and Yoshua Bengio · 2017
Earlier work this paper cites.
Spectrally-normalized margin bounds for neural networks
Peter L. Bartlett, Dylan J. Foster, and Matus Telgarsky · 2017
Earlier work this paper cites.
Size-independent sample complexity of neural networks
Noah Golowich, Alexander Rakhlin, and Ohad Shamir · 2017
Earlier work this paper cites.
Nearly-tight vc-dimension bounds for piecewise linear neural networks
Nick Harvey, Christopher Liaw, and Abbas Mehrabian · 2017
Earlier work this paper cites.
Implicit regularization in deep learning
Behnam Neyshabur · 2017
Cited alongside, same era.
Convolutional neural networks analyzed via convolutional sparse coding
Vardan Papyan, Yaniv Romano, and Michael Elad · 2017
Cited alongside, same era.
Opening the black box of deep neural networks via information
Ravid Shwartz-Ziv and Naftali Tishby · 2017
Cited alongside, same era.
Understanding deep learning requires rethinking generalization
Chiyuan Zhang, Samy Bengio, Moritz Hardt, Benjamin Recht, and Oriol Vinyals · 2017
Cited alongside, same era.
Dnn or k-nn: That is the generalize vs. memorize question, 2018
Gilad Cohen, Guillermo Sapiro, and Raja Giryes · 2018
Cited alongside, same era.
Sparsity-probe: Analysis tool for deep learning models
Ido Ben-Shaul and Shai Dekel · 2021
Later among the works it cites.
Evaluating large language models trained on code, 2021
Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott Gray, Nick Ryder, Mikhail Pavlov, Alethea Power, Lukasz Kaiser, Mohammad Bavarian, Clemens Winter, Philippe Tillet, Felipe Petroski Such, Dave Cummings, Matthias Plappert, Fotios Chantzis, Elizabeth Barnes, Ariel Herbert-Voss, William Hebgen Guss, Alex Nichol, Alex Paino, Nikolas Tezak, Jie Tang, Igor Babuschkin, Suchir Balaji, Shantanu Jain, William Saunders, Christopher Hesse, Andrew N. Carr, Jan Leike, Josh Achiam, Vedant Misra, Evan Morikawa, Alec Radford, Matthew Knight, Miles Brundage, Mira Murati, Katie Mayer, Peter Welinder, Bob McGrew, Dario Amodei, Sam McCandlish, Ilya Sutskever, and Wojciech Zaremba · 2021
Later among the works it cites.
Revealing the structure of deep neural networks via convex duality
Tolga Ergen and Mert Pilanci · 2021
Later among the works it cites.
Exploring deep neural networks via layer-peeled model: Minority collapse in imbalanced training
Cong Fang, Hangfeng He, Qi Long, and Weijie J. Su · 2021
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Characterizing implicit bias in terms of optimization geometry
Suriya Gunasekar, Jason Lee, Daniel Soudry, and Nathan Srebro · 2018
Cited alongside, same era.
A pac-bayesian approach to spectrally-normalized margin bounds for neural networks
Behnam Neyshabur, Srinadh Bhojanapalli, David A. McAllester, and Nathan Srebro · 2018
Cited alongside, same era.
A continuous-time view of early stopping for least squares regression
Alnur Ali, J. Zico Kolter, and Ryan J. Tibshirani · 2019
Cited alongside, same era.
BERT: Pre-training of deep bidirectional transformers for language understanding
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova · 2019
Cited alongside, same era.
Towards understanding the role of over-parametrization in generalization of neural networks
Behnam Neyshabur, Zhiyuan Li, Srinadh Bhojanapalli, Yann LeCun, and Nathan Srebro · 2019
Cited alongside, same era.
Language models are few-shot learners
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei · 2020
Cited alongside, same era.
Neural collapse with cross-entropy loss
Jianfeng Lu and Stefan Steinerberger · 2020
Cited alongside, same era.
Scaling vision transformers, 2021
Xiaohua Zhai, Alexander Kolesnikov, Neil Houlsby, and Lucas Beyer · 2021
Later among the works it cites.
A geometric analysis of neural collapse with unconstrained features
Zhihui Zhu, Tianyu DING, Jinxin Zhou, Xiao Li, Chong You, Jeremias Sulam, and Qing Qu · 2021
Later among the works it cites.
Nearest class-center simplification through intermediate layers
Ido Ben-Shaul and Shai Dekel · 2022
Closest in time.
Sgd noise and implicit low-rank bias in deep neural networks, 2022
Tomer Galanti and Tomaso Poggio · 2022
Closest in time.
Neural collapse under MSE loss: Proximity to and dynamics on the central path
X.Y. Han, Vardan Papyan, and David L. Donoho · 2022
Closest in time.
Limitations of neural collapse for understanding generalization in deep learning
Like Hui, Mikhail Belkin, and Preetum Nakkiran · 2022
Closest in time.
Training invariances and the low-rank phenomenon: beyond linear networks
Thien Le and Stefanie Jegelka · 2022
Closest in time.
Dynamics and neural collapse in deep classifiers trained with the square loss
Akshay Rangamani, Mengjia Xu, Andrzej Banburski, Qianli Liao, Tomer Galanti, and Tomaso Poggio · 2022
Closest in time.
Implicit regularization towards rank minimization in relu networks
Nadav Timor, Gal Vardi, and Ohad Shamir · 2022
Closest in time.
Extended unconstrained features model for exploring deep neural collapse, 2022
Tom Tirer and Joan Bruna · 2022
Closest in time.
Deepnet: Scaling transformers to 1,000 layers
Hongyu Wang, Shuming Ma, Li Dong, Shaohan Huang, Dongdong Zhang, and Furu Wei · 2022
Closest in time.
Image2point: 3d point-cloud understanding with 2d image pretrained models
Chenfeng Xu, Shijia Yang, Tomer Galanti, Bichen Wu, Xiangyu Yue, Bohan Zhai, Wei Zhan, Peter Vajda, Kurt Keutzer, and Masayoshi Tomizuka · 2022
Closest in time.