PREFER: Prompt Ensemble Learning via Feedback-Reflect-Refine

Understand

As an effective tool for eliciting the power of Large Language Models (LLMs), prompting has recently demonstrated unprecedented abilities across a variety of complex tasks.

• To further improve the performance, prompt ensemble has attracted substantial interest for tackling the hallucination and instability of LLMs.
• However, existing methods usually adopt a two-stage paradigm, which requires a pre-prepared set of prompts with substantial manual effort, and is unable to perform directed optimization for different weak learners.
• In this paper, we propose a simple, universal, and automatic method named PREFER (Pompt Ensemble learning via Feedback-Reflect-Refine) to address the stated limitations.