Fetching the paper…
Reading the bibliography…
State-of-the-art results on image recognition tasks are achieved using over-parameterized learning algorithms that (nearly) perfectly fit the training set and are known to fit well even random labels.
“High probability generalization bounds for uniformly stable algorithms with nearly optimal rate”
Vitaly Feldman and Jan Vondr“’ak · 1902
Earlier work this paper cites.
“Differential Privacy Has Disparate Impact on Model Accuracy”
Eugene Bagdasaryan and Vitaly Shmatikov · 1905
Earlier work this paper cites.
“What Do Compressed Deep Neural Networks Forget?”, 2020
Sara Hooker, Aaron Courville, Gregory Clark, Yann Dauphin and Andrea Frome · 1911
Earlier work this paper cites.
“Rademacher penalties and structural risk minimization”
Vladimir Koltchinskii · 1914
Earlier work this paper cites.
“Nearest neighbor pattern classification”
Thomas Cover and Peter Hart · 1967
Earlier work this paper cites.
“Distribution-free inequalities for the deleted and holdout error estimates”
Luc Devroye and Terry. Wagner · 1979
Earlier work this paper cites.
“Estimation of Dependences Based on Empirical Data”
V.. Vapnik · 1982
Earlier work this paper cites.
“How to construct random functions”
O. Goldreich, S. Goldwasser and S. Micali · 1986
Earlier work this paper cites.
“Support-vector networks”
Corinna Cortes and Vladimir Vapnik · 1995
Earlier work this paper cites.
“A decision-theoretic generalization of on-line learning and an application to boosting”
Y. Freund and R. Schapire · 1997
Earlier work this paper cites.
“Boosting the margin: a new explanation for the effectiveness of voting methods”
R. Schapire, Y. Freund, P. Bartlett and W. Lee · 1998
Earlier work this paper cites.
“Random forests”
Leo Breiman · 2001
Earlier work this paper cites.
“Stability and generalization”
Olivier Bousquet and Andr“’e Elisseeff · 2002
Earlier work this paper cites.
“Rademacher and Gaussian Complexities: Risk Bounds and Structural Results”
P. Bartlett and S. Mendelson · 2002
Earlier work this paper cites.
“A gentle introduction to concentration inequalities”
Karthik Sridharan · 2002
Earlier work this paper cites.
“Stability Results In Learning Theory”
Alexander Rakhlin, Sayan Mukherjee and Tomaso Poggio · 2005
Earlier work this paper cites.
“A new perspective on an old perceptron algorithm”
Shai Shalev-Shwartz and Yoram Singer · 2005
Earlier work this paper cites.
“Calibrating noise to sensitivity in private data analysis”
C. Dwork, F. McSherry, K. Nissim and A. Smith · 2006
Earlier work this paper cites.
“Learning theory: stability is sufficient for generalization and necessary and sufficient for consistency of empirical risk minimization”
Sayan Mukherjee, Partha Niyogi, Tomaso Poggio and Ryan Rifkin · 2006
Earlier work this paper cites.
Vitaly Feldman and Chiyuan Zhang · 2008
Earlier work this paper cites.
“ImageNet: A Large-Scale Hierarchical Image Database”
J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li and L. Fei-Fei · 2009
Earlier work this paper cites.
“Stochastic Convex Optimization”
S. Shalev-Shwartz, O. Shamir, N. Srebro and K. Sridharan · 2009
Earlier work this paper cites.
“Characterising Bias in Compressed Models”
Sara Hooker, Nyalleng Moorosi, Gregory Clark, Samy Bengio and Emily Denton · 2010
Earlier work this paper cites.
“Learnability, Stability and Uniform Convergence”
Shai Shalev-Shwartz, Ohad Shamir, Nathan Srebro and Karthik Sridharan · 2010
Earlier work this paper cites.
“Smoothness, Low Noise and Fast Rates”
Nathan Srebro, Karthik Sridharan and Ambuj Tewari · 2010
Earlier work this paper cites.
“Sun database: Large-scale scene recognition from abbey to zoo”
Jianxiong Xiao, James Hays, Krista Ehinger, Aude Oliva and Antonio Torralba · 2010
Earlier work this paper cites.
“Sample Complexity Bounds for Differentially Private Learning”
Kamalika Chaudhuri and Daniel Hsu · 2011
Earlier work this paper cites.
“When is Memorization of Irrelevant Training Data Necessary for High-Accuracy Learning?”
Gavin Brown, Mark Bun, Vitaly Feldman, Adam Smith and Kunal Talwar · 2012
Earlier work this paper cites.
“Extracting Training Data from Large Language Models”
Nicholas Carlini, Florian Tram“‘er, Eric Wallace, Matthew Jagielski, Ariel Herbert-Voss, Katherine Lee, Adam Roberts, Tom. Brown, Dawn Song, “’Ulfar Erlingsson, Alina Oprea and Colin Raffel · 2012
Cited alongside, same era.
“Explaining adaboost”
Robert Schapire · 2013
Cited alongside, same era.
“Rates of Convergence for Nearest Neighbor Classification”
Kamalika Chaudhuri and Sanjoy Dasgupta · 2014
Cited alongside, same era.
“The Algorithmic Foundations of Differential Privacy”
Cynthia Dwork and Aaron Roth · 2014
Cited alongside, same era.
“Preserving Statistical Validity in Adaptive Data Analysis” Extended abstract in STOC 2015
Cynthia Dwork, Vitaly Feldman, Moritz Hardt, Toniann Pitassi, Omer Reingold and Aaron Roth · 2014
Cited alongside, same era.
“Understanding deep learning requires rethinking generalization”
Chiyuan Zhang, Samy Bengio, Moritz Hardt, Benjamin Recht and Oriol Vinyals · 2017
Later among the works it cites.
“Overfitting or perfect fitting? risk bounds for classification and regression rules that interpolate”
Mikhail Belkin, Daniel Hsu and Partha Mitra · 2018
Later among the works it cites.
“To Understand Deep Learning We Need to Understand Kernel Learning”
Mikhail Belkin, Siyuan Ma and Soumik Mandal · 2018
Later among the works it cites.
“Does data interpolation contradict statistical optimality?”
Mikhail Belkin, Alexander Rakhlin and Alexandre Tsybakov · 2018
Later among the works it cites.
“Prototypical Examples in Deep Learning: Metrics, Characteristics, and Utility”, 2018
Nicholas Carlini, Ulfar Erlingsson and Nicolas Papernot · 2018
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
“Stochastic Gradient Descent, Weighted Sampling, and the Randomized Kaczmarz algorithm”
Deanna Needell, Rachel Ward and Nati Srebro · 2014
Cited alongside, same era.
“Capturing long-tail distributions of object subcategories”
Xiangxin Zhu, Dragomir Anguelov and Deva Ramanan · 2014
Cited alongside, same era.
“In Search of the Real Inductive Bias: On the Role of Implicit Regularization in Deep Learning”
Behnam Neyshabur, Ryota Tomioka and Nathan Srebro · 2015
Cited alongside, same era.
“Competitive distribution estimation: Why is good-turing good”
Alon Orlitsky and Ananda Suresh · 2015
Cited alongside, same era.
“Privacy-preserving deep learning”
Reza Shokri and Vitaly Shmatikov · 2015
Cited alongside, same era.
“Deep learning with differential privacy”
Martin Abadi, Andy Chu, Ian Goodfellow, H McMahan, Ilya Mironov, Kunal Talwar and Li Zhang · 2016
Cited alongside, same era.
Vitaly Feldman · 2016
Cited alongside, same era.
Gilad Cohen, Guillermo Sapiro and Raja Giryes · 2018
Later among the works it cites.
“Large Scale Fine-Grained Categorization and Domain-Specific Transfer Learning”
Yin Cui, Yang Song, Chen Sun, Andrew Howard and Serge Belongie · 2018
Later among the works it cites.
“Privacy-preserving Prediction”
Cynthia Dwork and Vitaly Feldman · 2018
Later among the works it cites.
“Algorithmic Regularization in Over-parameterized Matrix Sensing and Neural Networks with Quadratic Activations”
Yuanzhi Li, Tengyu Ma and Hongyang Zhang · 2018
Later among the works it cites.
“Understanding Membership Inferences on Well-Generalized Learning Models”
Yunhui Long, Vincent Bindschaedler, Lei Wang, Diyue Bu, Xiaofeng Wang, Haixu Tang, Carl. Gunter and Kai Chen · 2018
Later among the works it cites.
“Just interpolate: Kernel” ridgeless” regression can generalize”
Tengyuan Liang and Alexander Rakhlin · 2018
Later among the works it cites.
“The Power of Interpolation: Understanding the Effectiveness of SGD in Modern Over-parametrized Learning”
Siyuan Ma, Raef Bassily and Mikhail Belkin · 2018
Later among the works it cites.
“Learning Differentially Private Recurrent Language Models”
Brendan McMahan, Daniel Ramage, Kunal Talwar and Li Zhang · 2018
Later among the works it cites.
“The implicit bias of gradient descent on separable data”
Daniel Soudry, Elad Hoffer, Mor Nacson, Suriya Gunasekar and Nathan Srebro · 2018
Later among the works it cites.
“Towards demystifying membership inference attacks”
Stacey Truex, Ling Liu, Mehmet Gursoy, Lei Yu and Wenqi Wei · 2018
Later among the works it cites.
“The inaturalist species classification and detection dataset”
Grant Van, Oisin Mac, Yang Song, Yin Cui, Chen Sun, Alex Shepard, Hartwig Adam, Pietro Perona and Serge Belongie · 2018
Later among the works it cites.
“Benign Overfitting in Linear Regression”
Peter Bartlett, Philip Long, G“’abor Lugosi and Alexander Tsigler · 2019
Closest in time.
“Two models of double descent for weak features”
Mikhail Belkin, Daniel Hsu and Ji Xu · 2019
Closest in time.
“Differential privacy has disparate impact on model accuracy”
Eugene Bagdasaryan, Omid Poursaeed and Vitaly Shmatikov · 2019
Closest in time.
“Data scarcity, robustness and extreme multi-label classification”
Rohit Babbar and Bernhard Sch“”olkopf · 2019
Closest in time.
“Learning Imbalanced Datasets with Label-Distribution-Aware Margin Loss”
Kaidi Cao, Colin Wei, Adrien Gaidon, Nikos Arechiga and Tengyu Ma · 2019
Closest in time.
“The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networks”
Nicholas Carlini, Chang Liu, Jernej Kos, “’Ulfar Erlingsson and Dawn Song · 2019
Closest in time.
“Distribution Density, Tails, and Outliers in Machine Learning: Metrics and Applications”
Nicholas Carlini, “’Ulfar Erlingsson and Nicolas Papernot · 2019
Closest in time.
“Surprises in High-Dimensional Ridgeless Least Squares Interpolation”
Trevor Hastie, Andrea Montanari, Saharon Rosset and Ryan Tibshirani · 2019
Closest in time.
“Harmless interpolation of noisy data in regression”
Vidya Muthukumar, Kailas Vodrahalli and Anant Sahai · 2019
Closest in time.
“Consistency of Interpolation with Laplace Kernels is a High-Dimensional Phenomenon”
Alexander Rakhlin and Xiyu Zhai · 2019
Closest in time.
“Visualizations and Pretrained Models for “What Neural Networks Memorize and Why”.” https://pluskid.github.io/influence-memorization/ , 2020
Vitaly Feldman and Chiyuan Zhang · 2020
Closest in time.