Siamese CBOW: Optimizing Word Embeddings for Sentence Representations

Understand

We present the Siamese Continuous Bag of Words (Siamese CBOW) model, a neural network for efficient estimation of high-quality sentence embeddings.

• Averaging the embeddings of words in a sentence has proven to be a surprisingly successful and efficient way of obtaining sentence embeddings.
• However, word embeddings trained with the methods currently available are not optimized for the task of sentence representation, and, thus, likely to be suboptimal.
• Siamese CBOW handles this problem by training word embeddings directly for the purpose of being averaged.