Fetching the paper…

Reading the bibliography…

2022

Cracking White-box DNN Watermarks via Invariant Neuron Transforms

Yan, Yifan, Pan, Xudong, Wang, Yining et al.

Original
PDF
Post on X

Understand

Recently, how to protect the Intellectual Property (IP) of deep neural networks (DNN) becomes a major concern for the AI industry.

To combat potential model piracy, recent works explore various watermarking strategies to embed secret identity messages into the prediction behaviors or the internals (e.g., weights and neuron activation) of the target model.
Sacrificing less functionality and involving more knowledge about the target model, the latter branch of watermarking schemes (i.e., white-box model watermarking) is claimed to be accurate, credible and secure against most known watermark removal attacks, with emerging research efforts and applications in the industry.
In this paper, we present the first effective removal attack which cracks almost all the existing white-box watermarking schemes with provably no performance overhead and no required prior knowledge.

Cracking White-box DNN Watermarks via Invariant Neuron Transforms · Around

Built on

R. W. Hamming, “Error detecting and error correcting codes,” The Bell system technical journal , vol. 29, no. 2, pp. 147–160, 1950
1950
Earlier work this paper cites.
A. M. Chen, H.-m. Lu, and R. Hecht-Nielsen, “On the geometry of feedforward neural network error surfaces,” Neural Computation , vol. 5, no. 6, pp. 910–927, 1993
1993
Earlier work this paper cites.
J. Fridrich, “Image watermarking for tamper detection,” in Proceedings 1998 International Conference on Image Processing. ICIP98 (Cat. No. 98CB36269) , vol. 2. IEEE, 1998, pp. 404–408
1998
Earlier work this paper cites.
A. Radford, L. Metz, and S. Chintala, “Unsupervised representation learning with deep convolutional generative adversarial networks,” arXiv preprint arXiv:1511.06434 , 2015
Original
2015
Earlier work this paper cites.
G. Hinton, O. Vinyals, and J. Dean, “Distilling the knowledge in a neural network,” arXiv preprint arXiv:1503.02531 , 2015
Original
2015
Earlier work this paper cites.
B. Neyshabur, R. Salakhutdinov, and N. Srebro, “Path-sgd: Path-normalized optimization in deep neural networks,” arXiv preprint arXiv:1506.02617 , 2015
Original
2015
Earlier work this paper cites.
S. Ioffe and C. Szegedy, “Batch normalization: Accelerating deep network training by reducing internal covariate shift,” in International conference on machine learning . PMLR, 2015, pp. 448–456
2015
Earlier work this paper cites.
K. He, X. Zhang, S. Ren, and J. Sun, “Delving deep into rectifiers: Surpassing human-level performance on imagenet classification,” in Proceedings of the IEEE international conference on computer vision , 2015, pp. 1026–1034
2015
Earlier work this paper cites.
B. Xu, N. Wang, T. Chen, and M. Li, “Empirical evaluation of rectified activations in convolutional network,” arXiv preprint arXiv:1505.00853 , 2015
Original
2015
Earlier work this paper cites.
K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proceedings of the IEEE conference on computer vision and pattern recognition , 2016, pp. 770–778
2016
Earlier work this paper cites.
C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens, and Z. Wojna, “Rethinking the inception architecture for computer vision,” in Proceedings of the IEEE conference on computer vision and pattern recognition , 2016, pp. 2818–2826
2016
Earlier work this paper cites.
S. Zagoruyko and N. Komodakis, “Wide residual networks,” arXiv preprint arXiv:1605.07146 , 2016
Original
2016
Earlier work this paper cites.
A. See, M.-T. Luong, and C. D. Manning, “Compression of neural machine translation models via pruning,” arXiv preprint arXiv:1606.09274 , 2016
Original
2016
Earlier work this paper cites.
H. Li, A. Kadav, I. Durdanovic, H. Samet, and H. P. Graf, “Pruning filters for efficient convnets,” arXiv preprint arXiv:1608.08710 , 2016
Original
2016
Earlier work this paper cites.
J. L. Ba, J. R. Kiros, and G. E. Hinton, “Layer normalization,” arXiv preprint arXiv:1607.06450 , 2016
Original
2016
Earlier work this paper cites.
D. Ulyanov, A. Vedaldi, and V. Lempitsky, “Instance normalization: The missing ingredient for fast stylization,” arXiv preprint arXiv:1607.08022 , 2016
Original
2016
Earlier work this paper cites.
F. Tramèr, F. Zhang, A. Juels, M. K. Reiter, and T. Ristenpart, “Stealing machine learning models via prediction { \{ APIs } \} ,” in 25th USENIX security symposium (USENIX Security 16) , 2016, pp. 601–618
2016
Earlier work this paper cites.
R. Shokri, M. Stronati, C. Song, and V. Shmatikov, “Membership inference attacks against machine learning models,” in 2017 IEEE symposium on security and privacy (SP) . IEEE, 2017, pp. 3–18
2017
Earlier work this paper cites.
Y. Uchida, Y. Nagai, S. Sakazawa, and S. Satoh, “Embedding watermarks into deep neural networks,” in Proceedings of the 2017 ACM on International Conference on Multimedia Retrieval , 2017, pp. 269–277
2017
Earlier work this paper cites.
“Uchida,” https://github.com/yu4u/dnn-watermark , accessed: 2017-07-31
2017
Earlier work this paper cites.
T. Gu, B. Dolan-Gavitt, and S. Garg, “Badnets: Identifying vulnerabilities in the machine learning model supply chain,” arXiv preprint arXiv:1708.06733 , 2017
Original
2017
Earlier work this paper cites.
J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “Bert: Pre-training of deep bidirectional transformers for language understanding,” arXiv preprint arXiv:1810.04805 , 2018
Original
2018
Earlier work this paper cites.
B. Wang and N. Z. Gong, “Stealing hyperparameters in machine learning,” in 2018 IEEE Symposium on Security and Privacy (SP) . IEEE, 2018, pp. 36–52
2018
Earlier work this paper cites.

Similar

A. Salem, Y. Zhang, M. Humbert, P. Berrang, M. Fritz, and M. Backes, “Ml-leaks: Model and data independent membership inference attacks and defenses on machine learning models,” arXiv preprint arXiv:1806.01246 , 2018
Original
2018
Cited alongside, same era.
Y. Adi, C. Baum, M. Cisse, B. Pinkas, and J. Keshet, “Turning your weakness into a strength: Watermarking deep neural networks by backdooring,” in 27th { \{ USENIX } \} Security Symposium ( { \{ USENIX } \} Security 18) , 2018, pp. 1615–1631
2018
Cited alongside, same era.
J. Zhang, Z. Gu, J. Jang, H. Wu, M. P. Stoecklin, H. Huang, and I. Molloy, “Protecting intellectual property of deep neural networks with watermarking,” in Proceedings of the 2018 on Asia Conference on Computer and Communications Security , 2018, pp. 159–172
2018
Cited alongside, same era.
K. Ganju, Q. Wang, W. Yang, C. A. Gunter, and N. Borisov, “Property inference attacks on fully connected neural networks using permutation invariant representations,” in Proceedings of the 2018 ACM SIGSAC conference on computer and communications security , 2018, pp. 619–633

Then

H. Liu, Z. Weng, and Y. Zhu, “Watermarking deep neural networks with greedy residuals,” in International Conference on Machine Learning . PMLR, 2021, pp. 6978–6988
2021
Later among the works it cites.
L. Fan, K. W. Ng, C. S. Chan, and Q. Yang, “Deepip: Deep neural network intellectual property protection with passports,” IEEE Transactions on Pattern Analysis and Machine Intelligence , 2021
2021
Later among the works it cites.
D. S. Ong, C. S. Chan, K. W. Ng, L. Fan, and Q. Yang, “Protecting intellectual property of generative adversarial networks from ambiguity attacks,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2021, pp. 3630–3639
2021
Later among the works it cites.
X. Chen, T. Chen, Z. Zhang, and Z. Wang, “You are caught stealing my winning lottery ticket! making a lottery ticket claim its ownership,” Advances in Neural Information Processing Systems , vol. 34, 2021
2021
Later among the works it cites.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

2018

Cited alongside, same era.

Y. Wu and K. He, “Group normalization,” in Proceedings of the European conference on computer vision (ECCV) , 2018, pp. 3–19

2018

Cited alongside, same era.

K. Liu, B. Dolan-Gavitt, and S. Garg, “Fine-pruning: Defending against backdooring attacks on deep neural networks,” in International Symposium on Research in Attacks, Intrusions, and Defenses . Springer, 2018, pp. 273–294

2018

Cited alongside, same era.

E. Strubell, A. Ganesh, and A. McCallum, “Energy and policy considerations for deep learning in nlp,” arXiv preprint arXiv:1906.02243 , 2019

2019

Cited alongside, same era.

S. J. Oh, B. Schiele, and M. Fritz, “Towards reverse-engineering black-box neural networks,” in Explainable AI: Interpreting, Explaining and Visualizing Deep Learning . Springer, 2019, pp. 121–144

2019

Cited alongside, same era.

T. Orekondy, B. Schiele, and M. Fritz, “Knockoff nets: Stealing functionality of black-box models,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2019, pp. 4954–4963

2019

Cited alongside, same era.

B. Darvish Rouhani, H. Chen, and F. Koushanfar, “Deepsigns: An end-to-end watermarking framework for ownership protection of deep neural networks,” pp. 485–497, 2019

2019

Cited alongside, same era.

H. Chen, C. Fu, B. D. Rouhani, J. Zhao, and F. Koushanfar, “Deepattest: an end-to-end attestation framework for deep neural networks,” in 2019 ACM/IEEE 46th Annual International Symposium on Computer Architecture (ISCA) . IEEE, 2019, pp. 487–498

2019

Cited alongside, same era.

T. Wang and F. Kerschbaum, “Attacks on digital watermarks for deep neural networks,” in ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) . IEEE, 2019, pp. 2622–2626

2019

Cited alongside, same era.

S. Szyller, B. G. Atli, S. Marchal, and N. Asokan, “Dawn: Dynamic adversarial watermarking of neural networks,” in Proceedings of the 29th ACM International Conference on Multimedia , 2021, pp. 4417–4425

2021

Later among the works it cites.

H. Jia, C. A. Choquette-Choo, V. Chandrasekaran, and N. Papernot, “Entangled watermarks as a defense against model extraction,” in 30th { \{ USENIX } \} Security Symposium ( { \{ USENIX } \} Security 21) , 2021

2021

Later among the works it cites.

M. Shafieinejad, N. Lukas, J. Wang, X. Li, and F. Kerschbaum, “On the robustness of backdoor-based watermarking in deep neural networks,” in Proceedings of the 2021 ACM Workshop on Information Hiding and Multimedia Security , 2021, pp. 177–188

2021

Later among the works it cites.

X. Chen, W. Wang, C. Bender, Y. Ding, R. Jia, B. Li, and D. Song, “Refit: a unified watermark removal framework for deep learning systems with limited data,” in Proceedings of the 2021 ACM Asia Conference on Computer and Communications Security , 2021, pp. 321–335

2021

Later among the works it cites.

W. Aiken, H. Kim, S. Woo, and J. Ryoo, “Neural network laundering: Removing black-box backdoor watermarks from deep neural networks,” Computers & Security , vol. 106, p. 102277, 2021

2021

Later among the works it cites.

S. Guo, T. Zhang, H. Qiu, Y. Zeng, T. Xiang, and Y. Liu, “Fine-tuning is not enough: A simple yet effective watermark removal attack for dnn models,” in International Joint Conference on Artificial Intelligence (IJCAI) , 2021

2021

Later among the works it cites.

H. Jeong, D. Ryu, and J. Hur, “Neural network stealing via meltdown,” in 2021 International Conference on Information Networking (ICOIN) . IEEE, 2021, pp. 36–38

2021

Later among the works it cites.

“Riga,” https://github.com/TIANHAO-WANG/riga , accessed: 2021-2-6

2021

Later among the works it cites.

“Protection on the IPR

2021

Later among the works it cites.

“Greedy residuals,” https://github.com/eil/greedy-residuals , accessed: 2021-7-16

2021

Later among the works it cites.

“lottery verification,” https://github.com/VITA-Group/NO-stealing-LTH , accessed: 2021-10-1

2021

Later among the works it cites.

“Protection on the IPR

2021

Later among the works it cites.

“Deepipr,” https://github.com/kamwoh/DeepIPR , accessed: 2021-12-30

2021

Later among the works it cites.

“Passport-aware normalization,” https://github.com/ZJZAC/Passport-aware-Normalization , accessed: 2021-6-10

2021

Later among the works it cites.

J. H. Lim, C. S. Chan, K. W. Ng, L. Fan, and Q. Yang, “Protect, show, attend and tell: Empowering image captioning models with ownership protection,” Pattern Recognition , vol. 122, p. 108285, 2022

2022

Closest in time.

J. Chen, J. Wang, T. Peng, Y. Sun, P. Cheng, S. Ji, X. Ma, B. Li, and D. Song, “Copy, right? a testing framework for copyright protection of deep learning models,” in 2022 IEEE Symposium on Security and Privacy (SP) , 2022

2022

Closest in time.

N. Lukas, E. Jiang, X. Li, and F. Kerschbaum, “Sok: How robust is deep neural network image classification watermarking?” in 2022 IEEE Symposium on Security and Privacy (SP) , 2022

2022

Closest in time.

“Random permutation,” https://numpy.org/devdocs/reference/random/generated/numpy.random.permutation.html , accessed: 2022-1-10

2022

Closest in time.

K. Xu, J. Ba, R. Kiros, K. Cho, A. Courville, R. Salakhudinov, R. Zemel, and Y. Bengio, “Show, attend and tell: Neural image caption generation with visual attention,” in International conference on machine learning . PMLR, 2015, pp. 2048–2057

2057

Closest in time.