SLIM: Sparsified Late Interaction for Multi-Vector Retrieval with Inverted Indexes

Understand

This paper introduces Sparsified Late Interaction for Multi-vector (SLIM) retrieval with inverted indexes.

• Multi-vector retrieval methods have demonstrated their effectiveness on various retrieval datasets, and among them, ColBERT is the most established method based on the late interaction of contextualized token embeddings of pre-trained language models.
• However, efficient ColBERT implementations require complex engineering and cannot take advantage of off-the-shelf search libraries, impeding their practical use.
• To address this issue, SLIM first maps each contextualized token vector to a sparse, high-dimensional lexical space before performing late interaction between these sparse token embeddings.