Design of intentional backdoors in sequential models

Understand

Recent work has demonstrated robust mechanisms by which attacks can be orchestrated on machine learning models.

• In contrast to adversarial examples, backdoor or trojan attacks embed surgically modified samples with targeted labels in the model training process to cause the targeted model to learn to misclassify chosen samples in the presence of specific triggers, while keeping the model performance stable across other nominal samples.
• However, current published research on trojan attacks mainly focuses on classification problems, which ignores sequential dependency between inputs.
• In this paper, we propose methods to discreetly introduce and exploit novel backdoor attacks within a sequential decision-making agent, such as a reinforcement learning agent, by training multiple benign and malicious policies within a single long short-term memory (LSTM) network.