Post Training in Deep Learning with Last Kernel

Understand

One of the main challenges of deep learning methods is the choice of an appropriate training strategy.

• In particular, additional steps, such as unsupervised pre-training, have been shown to greatly improve the performances of deep structures.
• In this article, we propose an extra training step, called post-training, which only optimizes the last layer of the network.
• We show that this procedure can be analyzed in the context of kernel theory, with the first layers computing an embedding of the data and the last layer a statistical model to solve the task based on this embedding.