PIG: Privacy Jailbreak Attack on LLMs via Gradient-based Iterative In-Context Optimization

Understand

Large Language Models (LLMs) excel in various domains but pose inherent privacy risks.

• Existing methods to evaluate privacy leakage in LLMs often use memorized prefixes or simple instructions to extract data, both of which well-alignment models can easily block.
• Meanwhile, Jailbreak attacks bypass LLM safety mechanisms to generate harmful content, but their role in privacy scenarios remains underexplored.
• In this paper, we examine the effectiveness of jailbreak attacks in extracting sensitive information, bridging privacy leakage and jailbreak attacks in LLMs.