BadGPT: Exploring Security Vulnerabilities of ChatGPT via Backdoor Attacks to InstructGPT

Understand

Recently, ChatGPT has gained significant attention in research due to its ability to interact with humans effectively.

• The core idea behind this model is reinforcement learning (RL) fine-tuning, a new paradigm that allows language models to align with human preferences, i.e., InstructGPT.
• In this study, we propose BadGPT, the first backdoor attack against RL fine-tuning in language models.
• By injecting a backdoor into the reward model, the language model can be compromised during the fine-tuning stage.