Contextual Integrity in LLMs via Reasoning and Reinforcement Learning

Understand

As the era of autonomous agents making decisions on behalf of users unfolds, ensuring contextual integrity (CI) -- what is the appropriate information to share while carrying out a certain task -- becomes a central question to the field.

• We posit that CI demands a form of reasoning where the agent needs to reason about the context in which it is operating.
• To test this, we first prompt LLMs to reason explicitly about CI when deciding what information to disclose.
• We then extend this approach by developing a reinforcement learning (RL) framework that further instills in models the reasoning necessary to achieve CI.