ScalingNote: Scaling up Retrievers with Large Language Models for Real-World Dense Retrieval

Understand

Dense retrieval in most industries employs dual-tower architectures to retrieve query-relevant documents.

• Due to online deployment requirements, existing real-world dense retrieval systems mainly enhance performance by designing negative sampling strategies, overlooking the advantages of scaling up.
• Recently, Large Language Models (LLMs) have exhibited superior performance that can be leveraged for scaling up dense retrieval.
• However, scaling up retrieval models significantly increases online query latency.