2018

DeepSigns: A Generic Watermarking Framework for IP Protection of Deep Learning Models

Rouhani, Bita Darvish, Chen, Huili, Koushanfar, Farinaz

Understand

Deep Learning (DL) models have caused a paradigm shift in our ability to comprehend raw data in various important fields, ranging from intelligence warfare and healthcare to autonomous transportation and automated manufacturing.

A practical concern, in the rush to adopt DL models as a service, is protecting the models against Intellectual Property (IP) infringement.
The DL models are commonly built by allocating significant computational resources that process vast amounts of proprietary training data.
The resulting models are therefore considered to be the IP of the model builder and need to be protected to preserve the owner's competitive advantage.

Built on

I. J. Cox, J. Kilian, F. T. Leighton, and T. Shamoon, “Secure spread spectrum watermarking for multimedia,” IEEE transactions on image processing , vol. 6, no. 12, 1997
1997
Earlier work this paper cites.
Y. LeCun, C. Cortes, and C. J. Burges, “The mnist database of handwritten digits,” 1998
1998
Earlier work this paper cites.
F. Hartung and M. Kutter, “Multimedia watermarking techniques,” Proceedings of the IEEE , vol. 87, no. 7, 1999
1999
Earlier work this paper cites.
N. F. Johnson, Z. Duric, and S. Jajodia, Information Hiding: Steganography and Watermarking-Attacks and Countermeasures: Steganography and Watermarking: Attacks and Countermeasures . Springer Science & Business Media, 2001, vol. 1
2001
Earlier work this paper cites.
B. Furht and D. Kirovski, Multimedia security handbook . CRC press, 2004
2004
Earlier work this paper cites.
C.-S. Lu, Multimedia Security: Steganography and Digital Watermarking Techniques for Protection of Intellectual Property: Steganography and Digital Watermarking Techniques for Protection of Intellectual Property . Igi Global, 2004
2004
Earlier work this paper cites.
G. Qu and M. Potkonjak, Intellectual property protection in VLSI designs: theory and practice . Springer Science & Business Media, 2007
2007
Earlier work this paper cites.
A. Krizhevsky and G. Hinton, “Learning multiple layers of features from tiny images,” 2009
2009
Earlier work this paper cites.
L. Deng, D. Yu et al. , “Deep learning: methods and applications,” Foundations and Trends® in Signal Processing , vol. 7, no. 3–4, 2014
2014
Earlier work this paper cites.
I. J. Goodfellow, J. Shlens, and C. Szegedy, “Explaining and harnessing adversarial examples,” arXiv preprint arXiv:1412.6572 , 2014
Original
2014
Earlier work this paper cites.
K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” arXiv preprint arXiv:1409.1556 , 2014
Original
2014
Earlier work this paper cites.
J. T. Springenberg, A. Dosovitskiy, T. Brox, and M. Riedmiller, “Striving for simplicity: The all convolutional net,” arXiv preprint arXiv:1412.6806 , 2014
Original
2014
Earlier work this paper cites.

Similar

Y. N. Dauphin, R. Pascanu, C. Gulcehre, K. Cho, S. Ganguli, and Y. Bengio, “Identifying and attacking the saddle point problem in high-dimensional non-convex optimization,” in Advances in Neural Information Processing Systems (NIPS) , 2014
2014
Cited alongside, same era.
Y. LeCun, Y. Bengio, and G. Hinton, “Deep learning,” nature , vol. 521, no. 7553, 2015
2015
Cited alongside, same era.
M. Ribeiro, K. Grolinger, and M. A. Capretz, “Mlaas: Machine learning as a service,” in IEEE 14th International Conference on Machine Learning and Applications (ICMLA) , 2015
2015
Cited alongside, same era.
A. Choromanska, M. Henaff, M. Mathieu, G. B. Arous, and Y. LeCun, “The loss surfaces of multilayer networks,” in Artificial Intelligence and Statistics , 2015
2015
Cited alongside, same era.
S. Ioffe and C. Szegedy, “Batch normalization: Accelerating deep network training by reducing internal covariate shift,” arXiv preprint arXiv:1502.03167 , 2015
Original
2015
Cited alongside, same era.
A. B. Patel, T. Nguyen, and R. G. Baraniuk, “A probabilistic theory of deep learning,” arXiv preprint arXiv:1504.00641 , 2015
Original
2015
Cited alongside, same era.
S. Han, J. Pool, J. Tran, and W. Dally, “Learning both weights and connections for efficient neural network,” in Advances in Neural Information Processing Systems (NIPS) , 2015
2015
Cited alongside, same era.
S. Han, H. Mao, and W. J. Dally, “Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding,” arXiv preprint arXiv:1510.00149 , 2015
Original
2015
Cited alongside, same era.
M. Liang and X. Hu, “Recurrent convolutional neural network for object recognition,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , 2015
2015
Cited alongside, same era.
I. Goodfellow, Y. Bengio, A. Courville, and Y. Bengio, Deep learning . MIT press Cambridge, 2016, vol. 1
2016
Cited alongside, same era.
D. Lin, S. Talathi, and S. Annapureddy, “Fixed point quantization of deep convolutional networks,” in International Conference on Machine Learning (ICML) , 2016
2016
Cited alongside, same era.
S. Zagoruyko and N. Komodakis, “Wide residual networks,” arXiv preprint arXiv:1605.07146 , 2016
Original
2016
Cited alongside, same era.

Then

M. Abadi, P. Barham, J. Chen, Z. Chen, A. Davis, J. Dean, M. Devin, S. Ghemawat, G. Irving, M. Isard et al. , “Tensorflow: A system for large-scale machine learning.” in Usenix Symposium on Operating Systems Design and Implementation (OSDI) , vol. 16, 2016
2016
Later among the works it cites.
B. D. Rouhani, A. Mirhoseini, and F. Koushanfar, “Delight: Adding energy dimension to deep neural networks,” in Proceedings of the International Symposium on Low Power Electronics and Design (ISLPED) . ACM, 2016
2016
Later among the works it cites.
N. Tajbakhsh, J. Y. Shin, S. R. Gurudu, R. T. Hurst, C. B. Kendall, M. B. Gotway, and J. Liang, “Convolutional neural networks for medical image analysis: Full training or fine tuning?” IEEE transactions on medical imaging , vol. 35, no. 5, 2016
2016
Later among the works it cites.
Y. Uchida, Y. Nagai, S. Sakazawa, and S. Satoh, “Embedding watermarks into deep neural networks,” in Proceedings of the ACM on International Conference on Multimedia Retrieval , 2017
2017
Later among the works it cites.
E. L. Merrer, P. Perez, and G. Trédan, “Adversarial frontier stitching for remote neural network watermarking,” arXiv preprint arXiv:1711.01894 , 2017
Original
2017
Later among the works it cites.
K. Grosse, P. Manoharan, N. Papernot, M. Backes, and P. McDaniel, “On the (statistical) detection of adversarial examples,” arXiv preprint arXiv:1702.06280 , 2017
Original
2017
Later among the works it cites.
B. D. Rouhani, A. Mirhoseini, and F. Koushanfar, “Deep3: Leveraging three levels of parallelism for efficient deep learning,” in Proceedings of ACM 54th Annual Design Automation Conference (DAC) , 2017
2017
Later among the works it cites.
N. Papernot, P. McDaniel, I. Goodfellow, S. Jha, Z. B. Celik, and A. Swami, “Practical black-box attacks against machine learning,” in Proceedings of ACM on Asia Conference on Computer and Communications Security , 2017
2017
Later among the works it cites.
Y. Nagai, Y. Uchida, S. Sakazawa, and S. Satoh, “Digital watermarking for deep neural networks,” International Journal of Multimedia Information Retrieval , vol. 7, no. 1, 2018
2018
Closest in time.
Y. Adi, C. Baum, M. Cisse, B. Pinkas, and J. Keshet, “Turning your weakness into a strength: Watermarking deep neural networks by backdooring,” Usenix Security Symposium , 2018
2018
Closest in time.
B. D. Rouhani, M. Samragh, T. Javidi, and F. Koushanfar, “Safe machine learning and defeat-ing adversarial attacks,” IEEE Security and Privacy (S&P) Magazine , 2018
2018
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…