Enhancing Sentence Embedding with Generalized Pooling

Understand

Pooling is an essential component of a wide variety of sentence representation and embedding models.

• This paper explores generalized pooling methods to enhance sentence embedding.
• We propose vector-based multi-head attention that includes the widely used max pooling, mean pooling, and scalar self-attention as special cases.
• The model benefits from properly designed penalization terms to reduce redundancy in multi-head attention.