word2vec Explained: deriving Mikolov et al.'s negative-sampling word-embedding method

Understand

The word2vec software of Tomas Mikolov and colleagues (https://code.google.com/p/word2vec/ ) has gained a lot of traction lately, and provides state-of-the-art word embeddings.

• The learning models behind the software are described in two research papers.
• We found the description of the models in these papers to be somewhat cryptic and hard to follow.
• While the motivations and presentation may be obvious to the neural-networks language-modeling crowd, we had to struggle quite a bit to figure out the rationale behind the equations.