One Prompt is not Enough: Automated Construction of a Mixture-of-Expert Prompts

Understand

Large Language Models (LLMs) exhibit strong generalization capabilities to novel tasks when prompted with language instructions and in-context demos.

• Since this ability sensitively depends on the quality of prompts, various methods have been explored to automate the instruction design.
• While these methods demonstrated promising results, they also restricted the searched prompt to one instruction.
• Such simplification significantly limits their capacity, as a single demo-free instruction might not be able to cover the entire complex problem space of the targeted task.