Multimodal Word Distributions

Understand

Word embeddings provide point representations of words containing useful semantic information.

• We introduce multimodal word distributions formed from Gaussian mixtures, for multiple word meanings, entailment, and rich uncertainty information.
• To learn these distributions, we propose an energy-based max-margin objective.
• We show that the resulting approach captures uniquely expressive semantic information, and outperforms alternatives, such as word2vec skip-grams, and Gaussian embeddings, on benchmark datasets such as word similarity and entailment.