Towards Principled Unsupervised Learning

Understand

General unsupervised learning is a long-standing conceptual problem in machine learning.

• Supervised learning is successful because it can be solved by the minimization of the training error cost function.
• Unsupervised learning is not as successful, because the unsupervised objective may be unrelated to the supervised task of interest.
• For an example, density modelling and reconstruction have often been used for unsupervised learning, but they did not produced the sought-after performance gains, because they have no knowledge of the supervised tasks.