Measuring mutual information from finite data is difficult.
Reading the bibliography…
Asymptotic evaluation of certain markov process expectations for large time, iv
Donsker, M. and Varadhan, S. (1983) · 1983
Earlier work this paper cites.
Class-based n n -gram models of natural language
Brown, P. F., Desouza, P. V., Mercer, R. L., Pietra, V. J. D., and Lai, J. C. (1992) · 1992
Earlier work this paper cites.
An information-maximization approach to blind separation and blind deconvolution
Bell, A. J. and Sejnowski, T. J. (1995) · 1995
Earlier work this paper cites.
The information bottleneck method
Tishby, N., Pereira, F. C., and Bialek, W. (1999) · 1999
Earlier work this paper cites.
On the convergence rate of good-turing estimators
McAllester, D. A. and Schapire, R. E. (2000) · 2000
Earlier work this paper cites.
Always good turing: Asymptotically optimal probability estimation
Orlitsky, A., Santhanam, N., and Zhang1, J. (2003) · 2003
Earlier work this paper cites.
Estimating mutual information
Kraskov, A., Stögbauer, H., and Grassberger, P. (2004) · 2004
Earlier work this paper cites.
Estimating divergence functionals and the likelihood ratio by convex risk minimization
Nguyen, X., Wainwright, M. J., and Jordan, M. I. (2010) · 2010
Earlier work this paper cites.
Estimating the unseen: an n/log (n)-sample estimator for entropy and support size, shown optimal via new clts
Valiant, G. and Valiant, P. (2011) · 2011
Earlier work this paper cites.
A pac-bayesian tutorial with a dropout bound
McAllester, D. (2013) · 2013
Cited alongside, same era.
Efficient estimation of mutual information for strongly dependent variables
Gao, S., Ver Steeg, G., and Galstyan, A. (2015) · 2015
Cited alongside, same era.
Minimax estimation of discrete distributions under l 1 l_{1} loss
Han, Y., Jiao, J., and Weissman, T. (2015) · 2015
Cited alongside, same era.
Minimax estimation of functionals of discrete distributions
Jiao, J., Venkat, K., Han, Y., and Weissman, T. (2015) · 2015
Cited alongside, same era.
Nonparametric von mises estimators for entropies, divergences and mutual informations
Kandasamy, K., Krishnamurthy, A., Poczos, B., Wasserman, L., et al · 2015
Cited alongside, same era.
Effective approaches to attention-based neural machine translation
Luong, T., Pham, H., and Manning, C. D. (2015) · 2015
Cited alongside, same era.
Competitive distribution estimation: Why is good-turing good
Orlitsky, A. and Suresh, A. T. (2015) · 2015
Cited alongside, same era.
Who did what: A large-scale person-centered cloze dataset
Onishi, T., Wang, H., Bansal, M., Gimpel, K., and McAllester, D. (2016) · 2016
Cited alongside, same era.
Computing nonvacuous generalization bounds for deep (stochastic) neural networks with many more parameters than training data
Dziugaite, G. K. and Roy, D. M. (2017) · 2017
Later among the works it cites.
Do GANs learn the distribution? some theory and empirics
Arora, S., Risteski, A., and Zhang, Y. (2018) · 2018
Closest in time.
Mutual information neural estimation
Belghazi, M. I., Baratin, A., Rajeshwar, S., Ozair, S., Bengio, Y., Courville, A., and Hjelm, D. (2018) · 2018
Closest in time.
Information theoretic co-training
McAllester, D. (2018) · 2018
Closest in time.
Representation learning with contrastive predictive coding
Oord, A. v. d., Li, Y., and Vinyals, O. (2018) · 2018
Closest in time.
Learning deep representations by mutual information estimation and maximization
Hjelm, R. D., Fedorov, A., Lavoie-Marchildon, S., Grewal, K., Bachman, P., Trischler, A., and Bengio, Y. (2019) · 2019
Closest in time.
On variational bounds of mutual information
Poole, B., Ozair, S., Van Den Oord, A., Alemi, A., and Tucker, G. (2019) · 2019
Closest in time.
Mutual information maximization for simple and accurate part-of-speech induction
Stratos, K. (2019) · 2019
Closest in time.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…