Near-lossless Binarization of Word Embeddings

Understand

Word embeddings are commonly used as a starting point in many NLP models to achieve state-of-the-art performances.

• However, with a large vocabulary and many dimensions, these floating-point representations are expensive both in terms of memory and calculations which makes them unsuitable for use on low-resource devices.
• The method proposed in this paper transforms real-valued embeddings into binary embeddings while preserving semantic information, requiring only 128 or 256 bits for each vector.
• This leads to a small memory footprint and fast vector operations.