Towards Understanding Linear Word Analogies

Understand

A surprising property of word vectors is that word analogies can often be solved with vector arithmetic.

• However, it is unclear why arithmetic operators correspond to non-linear embedding models such as skip-gram with negative sampling (SGNS).
• We provide a formal explanation of this phenomenon without making the strong assumptions that past theories have made about the vector space and word distribution.
• Our theory has several implications.