Control Illusion: The Failure of Instruction Hierarchies in Large Language Models

Understand

Large language models (LLMs) are increasingly deployed with hierarchical instruction schemes, where certain instructions (e.g., system-level directives) are expected to take precedence over others (e.g., user messages).

• Yet, we lack a systematic understanding of how effectively these hierarchical control mechanisms work.
• We introduce a systematic evaluation framework based on constraint prioritization to assess how well LLMs enforce instruction hierarchies.
• Our experiments across six state-of-the-art LLMs reveal that models struggle with consistent instruction prioritization, even for simple formatting conflicts.