Benefits of Jointly Training Autoencoders: An Improved Neural Tangent Kernel Analysis

Understand

A remarkable recent discovery in machine learning has been that deep neural networks can achieve impressive performance (in terms of both lower training error and higher generalization capacity) in the regime where they are massively over-parameterized.

• Consequently, over the past year, the community has devoted growing interest in analyzing optimization and generalization properties of over-parameterized networks, and several breakthrough works have led to important theoretical progress.
• However, the majority of existing work only applies to supervised learning scenarios and hence are limited to settings such as classification and regression.
• In contrast, the role of over-parameterization in the unsupervised setting has gained far less attention.