Weight Poisoning Attacks on Pre-trained Models

Understand

Recently, NLP has seen a surge in the usage of large pre-trained models.

• Users download weights of models pre-trained on large datasets, then fine-tune the weights on a task of their choice.
• This raises the question of whether downloading untrusted pre-trained weights can pose a security threat.
• In this paper, we show that it is possible to construct ``weight poisoning'' attacks where pre-trained weights are injected with vulnerabilities that expose ``backdoors'' after fine-tuning, enabling the attacker to manipulate the model prediction simply by injecting an arbitrary keyword.