Generalizing Verifiable Instruction Following

Understand

A crucial factor for successful human and AI interaction is the ability of language models or chatbots to follow human instructions precisely.

• A common feature of instructions are output constraints like ``only answer with yes or no" or ``mention the word `abrakadabra' at least 3 times" that the user adds to craft a more useful answer.
• Even today's strongest models struggle with fulfilling such constraints.
• We find that most models strongly overfit on a small set of verifiable constraints from the benchmarks that test these abilities, a skill called precise instruction following, and are not able to generalize well to unseen output constraints.