Efficient Sentence Embedding using Discrete Cosine Transform

Understand

Vector averaging remains one of the most popular sentence embedding methods in spite of its obvious disregard for syntactic structure.

• While more complex sequential or convolutional networks potentially yield superior classification performance, the improvements in classification accuracy are typically mediocre compared to the simple vector averaging.
• As an efficient alternative, we propose the use of discrete cosine transform (DCT) to compress word sequences in an order-preserving manner.
• The lower order DCT coefficients represent the overall feature patterns in sentences, which results in suitable embeddings for tasks that could benefit from syntactic features.