2022

Threats to Pre-trained Language Models: Survey and Taxonomy

Guo, Shangwei, Xie, Chunlong, Li, Jiwei et al.

Understand

Pre-trained language models (PTLMs) have achieved great success and remarkable performance over a wide range of natural language processing (NLP) tasks.

However, there are also growing concerns regarding the potential security issues in the adoption of PTLMs.
In this survey, we comprehensively systematize recently discovered threats to PTLM systems and applications.
We perform our attack characterization from three interesting perspectives.

Built on

Introduction to the conll-2002 shared task: Language-independent named entity recognition
EF Tjong Kim Sang · 2002
Earlier work this paper cites.
SQuAD: 100,000+ questions for machine comprehension of text
Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang · 2016
Earlier work this paper cites.
Membership inference attacks against machine learning models
Reza Shokri, Marco Stronati, Congzheng Song, and Vitaly Shmatikov · 2017
Earlier work this paper cites.
Improving language understanding by generative pre-training
Alec Radford, Karthik Narasimhan, Tim Salimans, and Ilya Sutskever · 2018
Earlier work this paper cites.
GLUE: A multi-task benchmark and analysis platform for natural language understanding
Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel Bowman · 2018
Earlier work this paper cites.
Flair: An easy-to-use framework for state-of-the-art nlp
Alan Akbik, Tanja Bergmann, Duncan Blythe, Kashif Rasul, Stefan Schweter, and Roland Vollgraf · 2019
Earlier work this paper cites.
Robust neural machine translation with doubly adversarial inputs
Yong Cheng, Lu Jiang, and Wolfgang Macherey · 2019
Earlier work this paper cites.
A backdoor attack against lstm-based text classification systems
Jiazhu Dai, Chuanshuai Chen, and Yufeng Li · 2019
Earlier work this paper cites.
BERT: Pre-training of deep bidirectional transformers for language understanding
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova · 2019
Earlier work this paper cites.
Improving the robustness of question answering systems to question paraphrasing
Wee Chung Gan and Hwee Tou Ng · 2019
Earlier work this paper cites.
Thieves on sesame street! model extraction of BERT-based APIs
Kalpesh Krishna, Gaurav Singh Tomar, Ankur P Parikh, Nicolas Papernot, and Mohit Iyyer · 2019
Earlier work this paper cites.
Roberta: A robustly optimized bert pretraining approach
Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov · 2019
Earlier work this paper cites.
Robustness to modification with shared words in paraphrase identification
Zhouxing Shi and Minlie Huang · 2019
Earlier work this paper cites.
Xlnet: Generalized autoregressive pretraining for language understanding
Zhilin Yang, Zihang Dai, Yiming Yang, Jaime Carbonell, Russ R Salakhutdinov, and Quoc V Le · 2019
Earlier work this paper cites.
Paws: Paraphrase adversaries from word scrambling
Yuan Zhang, Jason Baldridge, and Luheng He · 2019
Earlier work this paper cites.
Badnl: Backdoor attacks against nlp models
Original
Xiaoyi Chen, Ahmed Salem, Michael Backes, Shiqing Ma, and Yang Zhang · 2020
Earlier work this paper cites.
What neural networks memorize and why: Discovering the long tail via influence estimation
Vitaly Feldman and Chiyuan Zhang · 2020
Earlier work this paper cites.
BAE: BERT-based adversarial examples for text classification
Siddhant Garg and Goutham Ramakrishnan · 2020
Earlier work this paper cites.
Exploring vulnerabilities of BERT-based APIs
Xuanli He, Lingjuan Lyu, Lichao Sun, Xiaojun Chang, and Jun Zhao · 2020
Earlier work this paper cites.

Similar

High accuracy and high fidelity extraction of neural networks
Matthew Jagielski, Nicholas Carlini, David Berthelot, Alex Kurakin, and Nicolas Papernot · 2020
Cited alongside, same era.
Is BERT really robust? a strong baseline for natural language attack on text classification and entailment
Di Jin, Zhijing Jin, Joey Tianyi Zhou, and Peter Szolovits · 2020
Cited alongside, same era.
The thieves on sesame street are polyglots-extracting multilingual models from monolingual APIs
Nitish Shirish Keskar, Bryan McCann, Caiming Xiong, and Richard Socher · 2020
Cited alongside, same era.
Weight poisoning attacks on pretrained models
Keita Kurita, Paul Michel, and Graham Neubig · 2020
Cited alongside, same era.
Contextualized perturbation for textual adversarial attack
Dianqi Li, Yizhe Zhang, Hao Peng, Liqun Chen, Chris Brockett, Ming-Ting Sun, and Bill Dolan · 2020
Cited alongside, same era.
BERT-ATTACK: Adversarial attack against BERT using BERT
Linyang Li, Ruotian Ma, Qipeng Guo, Xiangyang Xue, and Xipeng Qiu · 2020
Cited alongside, same era.
Adversarial training for large neural language models
Original
Xiaodong Liu, Hao Cheng, Pengcheng He, Weizhu Chen, Yu Wang, Hoifung Poon, and Jianfeng Gao · 2020
Cited alongside, same era.
Privacy risks of general-purpose language models
Xudong Pan, Mi Zhang, Shouling Ji, and Min Yang · 2020
Cited alongside, same era.
Pre-trained models for natural language processing: A survey
Xipeng Qiu, Tianxiang Sun, Yige Xu, Yunfan Shao, Ning Dai, and Xuanjing Huang · 2020
Cited alongside, same era.
Information leakage in embedding models
Congzheng Song and Ananth Raghunathan · 2020
Cited alongside, same era.
It’s Morphin’Time! combating linguistic discrimination with inflectional perturbations
Samson Tan, Shafiq Joty, Min-Yen Kan, and Richard Socher · 2020
Cited alongside, same era.
T3: Tree-autoencoder constrained adversarial text generation for targeted attack
Boxin Wang, Hengzhi Pei, Boyuan Pan, Qian Chen, Shuohang Wang, and Bo Li · 2020
Cited alongside, same era.

Then

Adversarial stylometry in the wild: Transferable lexical substitution attacks on author profiling
Chris Emmery, Ákos Kádár, and Grzegorz Chrupała · 2021
Later among the works it cites.
Defending against backdoor attacks in natural language generation
Chun Fan, Xiaoya Li, Yuxian Meng, Xiaofei Sun, Xiang Ao, Fei Wu, Jiwei Li, and Tianwei Zhang · 2021
Later among the works it cites.
Model extraction and adversarial transferability, your BERT is vulnerable!
Xuanli He, Lingjuan Lyu, Qiongkai Xu, and Lichao Sun · 2021
Later among the works it cites.
Backdoor attacks on pre-trained models by layerwise weight poisoning
Linyang Li, Demin Song, Xiaonan Li, Jiehang Zeng, Ruotian Ma, and Xipeng Qiu · 2021
Later among the works it cites.
Using adversarial attacks to reveal the statistical bias in machine reading comprehension models
Jieyu Lin, Jiajie Zou, and Nai Ding · 2021
Later among the works it cites.
A survey of transformers
Tianyang Lin, Yuxin Wang, Xiangyang Liu, and Xipeng Qiu · 2021
Later among the works it cites.
Generating natural language attacks in a hard label black box setting
Rishabh Maheshwary, Saket Maheshwary, and Vikram Pudi · 2021
Later among the works it cites.
Adv-OLM: Generating textual adversaries via OLM
Vijit Malik, Ashwani Bhat, and Ashutosh Modi · 2021
Later among the works it cites.
Dataset reconstruction attack against language models
Rrubaa Panchendrarajan and Suman Bhoi · 2021
Later among the works it cites.
ONION: A simple and effective defense against textual backdoor attacks
Fanchao Qi, Yangyi Chen, Mukai Li, Yuan Yao, Zhiyuan Liu, and Maosong Sun · 2021
Later among the works it cites.
Backdoor pre-trained models can transfer to all
Lujia Shen, Shouling Ji, Xuhong Zhang, Jinfeng Li, Jing Chen, Jie Shi, Chengfang Fang, Jianwei Yin, and Ting Wang · 2021
Later among the works it cites.
Why do pretrained language models help in downstream tasks? an analysis of head and prompt tuning
Colin Wei, Sang Michael Xie, and Tengyu Ma · 2021
Later among the works it cites.
Pre-trained models: Past, present and future
Han Xu, Zhang Zhengyan, Ding Ning, Gu Yuxian, Liu Xiao, Huo Yuqi, Qiu Jiezhong, Zhang Liang, Han Wentao, Huang Minlie, et al · 2021
Later among the works it cites.
Beyond model extraction: Imitation attack for black-box NLP APIs
Qiongkai Xu, Xuanli He, Lingjuan Lyu, Lizhen Qu, and Gholamreza Haffari · 2021
Later among the works it cites.
On the transferability of adversarial attacks against neural text classifier
Liping Yuan, Xiaoqing Zheng, Yi Zhou, Cho-Jui Hsieh, and Kai-Wei Chang · 2021
Later among the works it cites.
Grey-box extraction of natural language models
Santiago Zanella-Beguelin, Shruti Tople, Andrew Paverd, and Boris Köpf · 2021
Later among the works it cites.
Trojaning language models for fun and profit
Xinyang Zhang, Zheng Zhang, Shouling Ji, and Ting Wang · 2021
Later among the works it cites.
Red alarm for pre-trained models: Universal vulnerability to neuron-level backdoor attacks
Zhengyan Zhang, Guangxuan Xiao, Yongwei Li, Tian Lv, Fanchao Qi, Zhiyuan Liu, Yasheng Wang, Xin Jiang, and Maosong Sun · 2021
Later among the works it cites.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…