Learning from LDA using Deep Neural Networks

Understand

Latent Dirichlet Allocation (LDA) is a three-level hierarchical Bayesian model for topic inference.

• In spite of its great success, inferring the latent topic distribution with LDA is time-consuming.
• Motivated by the transfer learning approach proposed by~\newcite{hinton2015distilling}, we present a novel method that uses LDA to supervise the training of a deep neural network (DNN), so that the DNN can approximate the costly LDA inference with less computation.
• Our experiments on a document classification task show that a simple DNN can learn the LDA behavior pretty well, while the inference is speeded up tens or hundreds of times.