The Phantom Menace: Unmasking Privacy Leakages in Vision-Language Models

Understand

Vision-Language Models (VLMs) combine visual and textual understanding, rendering them well-suited for diverse tasks like generating image captions and answering visual questions across various domains.

• However, these capabilities are built upon training on large amount of uncurated data crawled from the web.
• The latter may include sensitive information that VLMs could memorize and leak, raising significant privacy concerns.
• In this paper, we assess whether these vulnerabilities exist, focusing on identity leakage.