APEER: Automatic Prompt Engineering Enhances Large Language Model Reranking

Understand

Large Language Models (LLMs) have significantly enhanced Information Retrieval (IR) across various modules, such as reranking.

• Despite impressive performance, current zero-shot relevance ranking with LLMs heavily relies on human prompt engineering.
• Existing automatic prompt engineering algorithms primarily focus on language modeling and classification tasks, leaving the domain of IR, particularly reranking, underexplored.
• Directly applying current prompt engineering algorithms to relevance ranking is challenging due to the integration of query and long passage pairs in the input, where the ranking complexity surpasses classification tasks.