Sparse Overcomplete Word Vector Representations

Understand

Current distributed representations of words show little resemblance to theories of lexical semantics.

• The former are dense and uninterpretable, the latter largely based on familiar, discrete classes (e.g., supersenses) and relations (e.g., synonymy and hypernymy).
• We propose methods that transform word vectors into sparse (and optionally binary) vectors.
• The resulting representations are more similar to the interpretable features typically used in NLP, though they are discovered automatically from raw corpora.