Efficient Non-parametric Estimation of Multiple Embeddings per Word in Vector Space

Understand

There is rising interest in vector-space word embeddings and their use in NLP, especially given recent methods for their fast estimation at very large scale.

• Nearly all this work, however, assumes a single vector per word type ignoring polysemy and thus jeopardizing their usefulness for downstream tasks.
• We present an extension to the Skip-gram model that efficiently learns multiple embeddings per word type.
• It differs from recent related work by jointly performing word sense discrimination and embedding learning, by non-parametrically estimating the number of senses per word type, and by its efficiency and scalability.