LLMs can be easily Confused by Instructional Distractions

Understand

Despite the fact that large language models (LLMs) show exceptional skill in instruction following tasks, this strength can turn into a vulnerability when the models are required to disregard certain instructions.

• Instruction-following tasks typically involve a clear task description and input text containing the target data to be processed.
• However, when the input itself resembles an instruction, confusion may arise, even if there is explicit prompting to distinguish between the task instruction and the input.
• We refer to this phenomenon as instructional distraction.