Mitigating the Influence of Distractor Tasks in LMs with Prior-Aware Decoding

Understand

The broad capabilities of Language Models (LMs) can be limited by their sensitivity to distractor tasks: LMs can infer secondary tasks from the prompt in addition to the intended one, leading to unwanted outputs.

• For example, prompt injection attacks can cause models to deviate from explicit directives.
• In some 'inverse scaling' cases, this unwanted behaviour actually worsens as models scale up to at least 540B parameters.
• We present a theoretical framework that interprets LMs as a product of experts that combine multiple data generation processes.