A Partial Break of the Honeypots Defense to Catch Adversarial Attacks

Understand

A recent defense proposes to inject "honeypots" into neural networks in order to detect adversarial attacks.

• We break the baseline version of this defense by reducing the detection true positive rate to 0\% and the detection AUC to 0.02, maintaining the original distortion bounds.
• The authors of the original paper have amended the defense in their CCS'20 paper to mitigate this attacks.
• To aid further research, we release the complete 2.5 hour keystroke-by-keystroke screen recording of our attack process at https://nicholas.carlini.com/code/ccs_honeypot_break.