Fast-Rate Loss Bounds via Conditional Information Measures with Applications to Neural Networks

B. Guedj, “A primer on PAC-Bayesian learning,” arXiv , Jan. 2019. [Online]. Available: http://arxiv.org/abs/1901.05353

Original

1901

Earlier work this paper cites.

A. Esposito, M. Gastpar, and I. Issa, “Generalization error bounds via Rènyi f f -divergences and maximal leakage,” arXiv , Dec. 2019. [Online]. Available: http://arxiv.org/abs/1912.01439

Original

1912

Earlier work this paper cites.

D. McAllester, “Some PAC-Bayesian theorems,” in Proc. Conf. Learn. Theory (COLT) , Madison, WI, July 1998

1998

Earlier work this paper cites.

T. Steinke and L. Zakynthinou, “Reasoning about generalization via conditional mutual information,” in Proc. Conf. Learn. Theory (COLT) , Graz, Austria, July 2020. [Online]. Available: https://arxiv.org/abs/2001.09122

Original

2001

Earlier work this paper cites.

M. Haghifam, J. Negrea, A. Khisti, D. Roy, and G. Dziugaite, “Sharpened generalization bounds based on conditional mutual information and an application to noisy, iterative algorithms,” arXiv , Apr. 2020. [Online]. Available: http://arxiv.org/abs/2004.12983

Original

2004

Earlier work this paper cites.

G. Dziugaite, K. Hsu, W. Gharbieh, and D. Roy, “On the role of data in PAC-Bayes bounds,” June 2020. [Online]. Available: https://arxiv.org/abs/2006.10929

Original

2006

Earlier work this paper cites.

O. Catoni, PAC-Bayesian Supervised Classification: The Thermodynamics of Statistical Learning . IMS Lecture Notes Monogr. Ser., 2007, vol. 56

2007

Earlier work this paper cites.

D. McAllester, “A PAC-Bayesian tutorial with a dropout bound,” July 2013. [Online]. Available: http://arxiv.org/abs/1307.2118

Original

2013

Cited alongside, same era.

D. Russo and J. Zou, “Controlling bias in adaptive data analysis using information theory,” in Proc. Artif. Intell. Statist. (AISTATS) , Cadiz, Spain, May 2016

2016

Cited alongside, same era.

A. Xu and M. Raginsky, “Information-theoretic analysis of generalization capability of learning algorithms,” in Proc. Conf. Neural Inf. Process. Syst. (NeurIPS) , Long Beach, CA, Dec. 2017

2017

Cited alongside, same era.

G. Dziugaite and D. Roy, “Computing nonvacuous generalization bounds for deep (stochastic) neural networks with many more parameters than training data,” in Proc. Conf. Uncertainty in Artif. Intell. (UAI) , Sydney, Australia, Aug. 2017

2017

Cited alongside, same era.

J. Negrea, M. Haghifam, G. Dziugaite, A. Khisti, and D. Roy, “Information-theoretic generalization bounds for SGLD via data-dependent estimates,” in Proc. Conf. Neural Inf. Process. Syst. (NeurIPS) , Vancouver, Canada, Dec. 2019

2019

Later among the works it cites.

J. Yang, S. Sun, and D. M. Roy, “Fast-rate PAC-Bayes generalization bounds via shifted Rademacher processes,” in Proc. Conf. Neural Inf. Process. Syst. (NeurIPS) , Vancouver, Canada, Dec. 2019

2019

Later among the works it cites.

Y. Polyanskiy and Y. Wu, Lecture Notes On Information Theory , 2019. [Online]. Available: http://www.stat.yale.edu/%7Eyw562/teaching/itlectures.pdf

2019

Later among the works it cites.

M. J. Wainwright, High-Dimensional Statistics: a Non-Asymptotic Viewpoint . Cambridge, U.K.: Cambridge Univ. Press, 2019

2019

Later among the works it cites.

W. Zhou, V. Veitch, M. Austern, R. Adams, and P. Orbanz, “Non-vacuous generalization bounds at the ImageNet scale: a PAC-Bayesian compression approach,” in Proc. Int. Conf. Learn. Representations (ICLR) , New Orleans, LA, May 2019