Information-theoretic analysis of generalization capability of learning algorithms

K. L. Buescher and P. R. Kumar, “Learning by canonical smooth estimation. I. Simultaneous estimation,” IEEE Transactions on Automatic Control , vol. 41, no. 4, pp. 545–556, Apr 1996

1996

Earlier work this paper cites.

L. Devroye, L. Györfi, and G. Lugosi, A Probabilistic Theory of Pattern Recognition . Springer, 1996

1996

Earlier work this paper cites.

S. Verdú, “The exponential distribution in information theory,” Problems of Information Transmission , vol. 32, no. 1, pp. 86–95, 1996

Original

1996

Earlier work this paper cites.

O. Bousquet and A. Elisseeff, “Stability and generalization,” J. Machine Learning Res. , vol. 2, pp. 499–526, 2002

2002

Earlier work this paper cites.

S. Boucheron, O. Bousquet, and G. Lugosi, “Theory of classification: a survey of some recent advances,” ESAIM: Probability and Statistics , vol. 9, pp. 323–375, 2005

2005

Earlier work this paper cites.

T. Zhang, “Information-theoretic upper and lower bounds for statistical estimation,” IEEE Trans. Inform. Theory , vol. 52, no. 4, pp. 1307 – 1321, 2006

2006

Earlier work this paper cites.

F. McSherry and K. Talwar, “Mechanism design via differential privacy,” in Proceedings of 48th Annual IEEE Symposium on Foundations of Computer Science (FOCS) , 2007

2007

Earlier work this paper cites.

S. Shalev-Shwartz, O. Shamir, N. Srebro, and K. Sridharan, “Learnability, stability and uniform convergence,” J. Mach. Learn. Res. , vol. 11, pp. 2635–2670, 2010

2010

Earlier work this paper cites.

S. Boucheron, G. Lugosi, and P. Massart, Concentration Inequalities: A Nonasymptotic Theory of Independence . Oxford Univ. Press, 2013

2013

Cited alongside, same era.

S. Shalev-Shwartz and S. Ben-David, Understanding Machine Learning: From Theory to Algorithms . Cambridge University Press, 2014

2014

Cited alongside, same era.

C. Dwork and A. Roth, “The algorithmic foundations of differential privacy,” Foundations and Trends in Theoretical Computer Science , vol. 9, no. 3-4, 2014

2014

Cited alongside, same era.

C. Dwork, V. Feldman, M. Hardt, T. Pitassi, O. Reingold, and A. Roth, “Preserving statistical validity in adaptive data analysis,” in Proc. of 47th ACM Symposium on Theory of Computing (STOC) , 2015

2015

Cited alongside, same era.

——, “Generalization in adaptive data analysis and holdout reuse,” in 28th Annual Conference on Neural Information Processing Systems (NIPS) , 2015

Y.-X. Wang, J. Lei, and S. E. Fienberg, “On-average kl-privacy and its equivalence to generalization for max-entropy mechanisms,” in Proceedings of the International Conference on Privacy in Statistical Databases , 2016

2016

Later among the works it cites.

M. Raginsky, A. Rakhlin, M. Tsao, Y. Wu, and A. Xu, “Information-theoretic analysis of stability and bias of learning algorithms,” in Proceedings of IEEE Information Theory Workshop , 2016

2016

Later among the works it cites.

M. Raginsky, “Strong data processing inequalities and Φ \Phi -Sobolev inequalities for discrete channels,” IEEE Trans. Inform. Theory , vol. 62, no. 6, pp. 3355–3389, 2016

2016

Later among the works it cites.

I. Goodfellow, Y. Bengio, and A. Courville, Deep Learning . MIT Press, 2016

2016

Later among the works it cites.