Robust Encodings: A Framework for Combating Adversarial Typos

Understand

Despite excellent performance on many tasks, NLP systems are easily fooled by small adversarial perturbations of inputs.

• Existing procedures to defend against such perturbations are either (i) heuristic in nature and susceptible to stronger attacks or (ii) provide guaranteed robustness to worst-case attacks, but are incompatible with state-of-the-art models like BERT.
• In this work, we introduce robust encodings (RobEn): a simple framework that confers guaranteed robustness, without making compromises on model architecture.
• The core component of RobEn is an encoding function, which maps sentences to a smaller, discrete space of encodings.