CBOW Is Not All You Need: Combining CBOW with the Compositional Matrix Space Model

Understand

Continuous Bag of Words (CBOW) is a powerful text embedding method.

• Due to its strong capabilities to encode word content, CBOW embeddings perform well on a wide range of downstream tasks while being efficient to compute.
• However, CBOW is not capable of capturing the word order.
• The reason is that the computation of CBOW's word embeddings is commutative, i.e., embeddings of XYZ and ZYX are the same.