List-aware Reranking-Truncation Joint Model for Search and Retrieval-augmented Generation

Understand

The results of information retrieval (IR) are usually presented in the form of a ranked list of candidate documents, such as web search for humans and retrieval-augmented generation for large language models (LLMs).

• List-aware retrieval aims to capture the list-level contextual features to return a better list, mainly including reranking and truncation.
• Reranking finely re-scores the documents in the list.
• Truncation dynamically determines the cut-off point of the ranked list to achieve the trade-off between overall relevance and avoiding misinformation from irrelevant documents.