A Contrastive Pre-training Approach to Learn Discriminative Autoencoder for Dense Retrieval

Understand

Dense retrieval (DR) has shown promising results in information retrieval.

• In essence, DR requires high-quality text representations to support effective search in the representation space.
• Recent studies have shown that pre-trained autoencoder-based language models with a weak decoder can provide high-quality text representations, boosting the effectiveness and few-shot ability of DR models.
• However, even a weak autoregressive decoder has the bypass effect on the encoder.