Semantic Membership Inference Attack against Large Language Models

Understand

Membership Inference Attacks (MIAs) determine whether a specific data point was included in the training set of a target model.

• In this paper, we introduce the Semantic Membership Inference Attack (SMIA), a novel approach that enhances MIA performance by leveraging the semantic content of inputs and their perturbations.
• SMIA trains a neural network to analyze the target model's behavior on perturbed inputs, effectively capturing variations in output probability distributions between members and non-members.
• We conduct comprehensive evaluations on the Pythia and GPT-Neo model families using the Wikipedia dataset.