Fetching the paper…
Reading the bibliography…
Understanding the informative behaviour of deep neural networks is challenged by misused estimators and the complexity of network structure, which leads to inconsistent observations and diversified interpretation.
On measures of entropy and information
Alfréd Rényi et al · 1961
Earlier work this paper cites.
Determinant inequalities via information theory
Thomas M Cover and A Thomas · 1988
Earlier work this paper cites.
Entropy expressions and their estimators for multivariate distributions
Nabil Ali Ahmed and DV Gokhale · 1989
Earlier work this paper cites.
Elements of information theory
Thomas M Cover · 1999
Earlier work this paper cites.
The information bottleneck method
Naftali Tishby, Fernando C Pereira, and William Bialek · 2000
Earlier work this paper cites.
Estimating mutual information
Alexander Kraskov, Harald Stögbauer, and Peter Grassberger · 2004
Earlier work this paper cites.
Estimation of the entropy of a multivariate normal distribution
Neeraj Misra, Harshinder Singh, and Eugene Demchuk · 2005
Earlier work this paper cites.
Segmentation of multivariate mixed data via lossy data coding and compression
Yi Ma, Harm Derksen, Wei Hong, and John Wright · 2007
Earlier work this paper cites.
Classification via minimum incremental coding length (micl)
John Wright, Yi Ma, Yangyu Tao, Zhuochen Lin, and Heung-Yeung Shum · 2007
Earlier work this paper cites.
Bayesian estimation of the entropy of the multivariate gaussian
Santosh Srivastava and Maya R Gupta · 2008
Earlier work this paper cites.
Parametric bayesian estimation of differential entropy and relative entropy
Maya Gupta and Santosh Srivastava · 2010
Earlier work this paper cites.
Nonparametric estimation of conditional information and divergences
Barnabás Póczos and Jeff Schneider · 2012
Earlier work this paper cites.
Jensen-bregman logdet divergence with application to efficient similarity search for covariance matrices
Anoop Cherian, Suvrit Sra, Arindam Banerjee, and Nikolaos Papanikolopoulos · 2012
Earlier work this paper cites.
Information-theoretic measures of influence based on content dynamics
Greg Ver Steeg and Aram Galstyan · 2013
Earlier work this paper cites.
Measures of entropy from data using infinitely divisible kernels
Luis Gonzalo Sanchez Giraldo, Murali Rao, and Jose C Principe · 2014
Earlier work this paper cites.
Inequalities with determinants of perturbed positive matrices
Ivan Matic · 2014
Cited alongside, same era.
Log-determinant divergences revisited: Alpha-beta and gamma log-det divergences
Andrzej Cichocki, Sergio Cruces, and Shun-ichi Amari · 2015
Cited alongside, same era.
Law of log determinant of sample covariance matrix and optimal estimation of differential entropy for high-dimensional gaussian distributions
T Tony Cai, Tengyuan Liang, and Harrison H Zhou · 2015
Cited alongside, same era.
Strong subadditivity for log-determinant of covariance matrices and its applications
Gerardo Adesso and R Simon · 2016
Cited alongside, same era.
Opening the black box of deep neural networks via information
Ravid Shwartz-Ziv and Naftali Tishby · 2017
Cited alongside, same era.
Understanding autoencoders with information theoretic concepts
Shujian Yu and Jose C Principe · 2019
Later among the works it cites.
Adaptive estimators show information compression in deep neural networks
Ivan Chelombiev, Conor Houghton, and Cian O’Donnell · 2019
Later among the works it cites.
Entropy and mutual information in models of deep neural networks
Marylou Gabrié, Andre Manoel, Clément Luneau, Jean Barbier, Nicolas Macris, Florent Krzakala, and Lenka Zdeborová · 2019
Later among the works it cites.
Do compressed representations generalize better?
Hassan Hafez-Kolahi, Shohreh Kasaei, and Mahdiyeh Soleymani-Baghshah · 2019
Later among the works it cites.
Learning representations for neural network-based classification using the information bottleneck principle
Rana Ali Amjad and Bernhard C Geiger · 2019
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Nonparanormal information estimation
Shashank Singh and Barnabás Póczos · 2017
Cited alongside, same era.
From log-determinant inequalities to gaussian entanglement via recoverability theory
Ludovico Lami, Christoph Hirche, Gerardo Adesso, and Andreas Winter · 2017
Cited alongside, same era.
A comparison of methods for estimating the determinant of high-dimensional covariance matrix
Zongliang Hu, Kai Dong, Wenlin Dai, and Tiejun Tong · 2017
Cited alongside, same era.
Estimating information flow in deep neural networks
Ziv Goldfeld, Ewout van den Berg, Kristjan Greenewald, Igor Melnyk, Nam Nguyen, Brian Kingsbury, and Yury Polyanskiy · 2018
Cited alongside, same era.
Mine: mutual information neural estimation
Mohamed Ishmael Belghazi, Aristide Baratin, Sai Rajeswar, Sherjil Ozair, Yoshua Bengio, Aaron Courville, and R Devon Hjelm · 2018
Cited alongside, same era.
Evaluating capability of deep neural networks for image classification via information plane
Hao Cheng, Dongze Lian, Shenghua Gao, and Yanlin Geng · 2018
Cited alongside, same era.
Utilizing information bottleneck to evaluate the capability of deep neural networks for image classification
Hao Cheng, Dongze Lian, Shenghua Gao, and Yanlin Geng · 2019
Cited alongside, same era.
On information plane analyses of neural network classifiers–a review
Bernhard C Geiger · 2020
Later among the works it cites.
Understanding convolutional neural networks with information theory: An initial exploration
Shujian Yu, Kristoffer Wickstrøm, Robert Jenssen, and José C Príncipe · 2020
Later among the works it cites.
On the information plane of autoencoders
Nicolás I Tapia and Pablo A Estévez · 2020
Later among the works it cites.
Deep networks from the principle of rate reduction
Kwan Ho Ryan Chan, Yaodong Yu, Chong You, Haozhi Qi, John Wright, and Yi Ma · 2020
Later among the works it cites.
Learning diverse and discriminative representations via the principle of maximal coding rate reduction
Yaodong Yu, Kwan Ho Ryan Chan, Chong You, Chaobing Song, and Yi Ma · 2020
Later among the works it cites.
Incremental learning via rate reduction
Ziyang Wu, Christina Baek, Chong You, and Yi Ma · 2020
Later among the works it cites.
High-dimensional covariance matrix estimation
Clifford Lam · 2020
Later among the works it cites.
Measuring dependence with matrix-based entropy functional
Shujian Yu, Francesco Alesiani, Xi Yu, Robert Jenssen, and Jose C Principe · 2021
Closest in time.
Information bottleneck theory on convolutional neural networks
Junjie Li and Ding Liu · 2021
Closest in time.
Deep deterministic information bottleneck with matrix-based entropy functional
Xi Yu, Shujian Yu, and Jose C Principe · 2021
Closest in time.