Compressing Word Embeddings via Deep Compositional Code Learning

Understand

Natural language processing (NLP) models often require a massive number of parameters for word embeddings, resulting in a large storage or memory footprint.

• Deploying neural NLP models to mobile devices requires compressing the word embeddings without any significant sacrifices in performance.
• For this purpose, we propose to construct the embeddings with few basis vectors.
• For each word, the composition of basis vectors is determined by a hash code.