Modeling Documents with Deep Boltzmann Machines

Understand

We introduce a Deep Boltzmann Machine model suitable for modeling and extracting latent semantic representations from a large unstructured collection of documents.

• We overcome the apparent difficulty of training a DBM with judicious parameter tying.
• This parameter tying enables an efficient pretraining algorithm and a state initialization scheme that aids inference.
• The model can be trained just as efficiently as a standard Restricted Boltzmann Machine.