PACIT: Unlocking the Power of Examples for Better In-Context Instruction Tuning

Understand

Instruction tuning enhances the instruction following ability of large language models by finetuning with supervised instruction data.

• Previous work proposes in-context instruction tuning (ICIT) where specific positive or negative examples are incorporated into the prompt for better performance.
• In this work, we propose PACIT, a simple and effective in-context instruction tuning method, inspired by the pedagogical concept of desirable difficulty.
• The PACIT method unlocks the power of examples by encouraging the model to actively learn to grasp the distinctions between the positive and negative examples instead of merely reading.