Spinning Sequence-to-Sequence Models with Meta-Backdoors

Understand

We investigate a new threat to neural sequence-to-sequence (seq2seq) models: training-time attacks that cause models to "spin" their output and support a certain sentiment when the input contains adversary-chosen trigger words.

• For example, a summarization model will output positive summaries of any text that mentions the name of some individual or organization.
• We introduce the concept of a "meta-backdoor" to explain model-spinning attacks.
• These attacks produce models whose output is valid and preserves context, yet also satisfies a meta-task chosen by the adversary (e.g., positive sentiment).