Information-Theoretic Foundations for Machine Learning

Understand

The progress of machine learning over the past decade is undeniable.

• In retrospect, it is both remarkable and unsettling that this progress was achievable with little to no rigorous theory to guide experimentation.
• Despite this fact, practitioners have been able to guide their future experimentation via observations from previous large-scale empirical investigations.
• In this work, we propose a theoretical framework which attempts to provide rigor to existing practices in machine learning.