2020

DarKnight: A Data Privacy Scheme for Training and Inference of Deep Neural Networks

Hashemi, Hanieh, Wang, Yongqin, Annavaram, Murali

Understand

Protecting the privacy of input data is of growing importance as machine learning methods reach new application domains.

In this paper, we provide a unified training and inference framework for large DNNs while protecting input privacy and computation integrity.
Our approach called DarKnight uses a novel data blinding strategy using matrix masking to create input obfuscation within a trusted execution environment (TEE).
Our rigorous mathematical proof demonstrates that our blinding process provides information-theoretic privacy guarantee by bounding information leakage.

Built on

Privacy-preserving inference in machine learning services using trusted execution environments
Original
Krishna Giri Narra, Zhifeng Lin, Yongqin Wang, Keshav Balasubramaniam, and Murali Annavaram · 1912
Earlier work this paper cites.
Suppression methodology and statistical disclosure control
Lawrence H Cox · 1980
Earlier work this paper cites.
The confidentiality and analytic usefulness of masked business microdata
Nancy Spruill · 1983
Earlier work this paper cites.
A method for limiting disclosure in microdata based on random noise and transformation
Jay J Kim · 1986
Earlier work this paper cites.
Matrix masking methods for disclosure limitation in microdata
LH Cox · 1994
Earlier work this paper cites.
Elements of information theory
Thomas M Cover · 1999
Earlier work this paper cites.
Trustzone: Integrated hardware and software security
Tiago Alves · 2004
Earlier work this paper cites.
Fully homomorphic encryption using ideal lattices
Craig Gentry · 2009
Earlier work this paper cites.
Learning multiple layers of features from tiny images
Alex Krizhevsky, Geoffrey Hinton, et al · 2009
Earlier work this paper cites.
Disaggregated memory for expansion and sharing in blade servers
Kevin Lim, Jichuan Chang, Trevor Mudge, Parthasarathy Ranganathan, Steven K Reinhardt, and Thomas F Wenisch · 2009
Earlier work this paper cites.
Data confidentiality: A review of methods for statistical disclosure limitation and methods for assessing privacy
Gregory J Matthews, Ofer Harel, et al · 2011
Earlier work this paper cites.
Rappor: Randomized aggregatable privacy-preserving ordinal response
Úlfar Erlingsson, Vasyl Pihur, and Aleksandra Korolova · 2014
Earlier work this paper cites.
Very deep convolutional networks for large-scale image recognition
Original
Karen Simonyan and Andrew Zisserman · 2014
Earlier work this paper cites.
System and method for predicting behaviors of detected objects, February 25 2014
Jiajun Zhu, David I Ferguson, and Dmitri A Dolgov · 2014
Earlier work this paper cites.
Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding
Original
Song Han, Huizi Mao, and William J Dally · 2015
Earlier work this paper cites.
Imagenet large scale visual recognition challenge
Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, et al · 2015
Earlier work this paper cites.
Privacy-preserving deep learning
Reza Shokri and Vitaly Shmatikov · 2015
Earlier work this paper cites.
Controlled-channel attacks: Deterministic side channels for untrusted operating systems
Yuanzhong Xu, Weidong Cui, and Marcus Peinado · 2015
Earlier work this paper cites.
Deep learning with differential privacy
Martin Abadi, Andy Chu, Ian Goodfellow, H Brendan McMahan, Ilya Mironov, Kunal Talwar, and Li Zhang · 2016
Earlier work this paper cites.
An analysis of deep neural network models for practical applications
Original
Alfredo Canziani, Adam Paszke, and Eugenio Culurciello · 2016
Earlier work this paper cites.
Intel sgx explained
Victor Costan and Srinivas Devadas · 2016
Earlier work this paper cites.

Similar

Sanctum: Minimal hardware extensions for strong software isolation
Victor Costan, Ilia Lebedev, and Srinivas Devadas · 2016
Cited alongside, same era.
Learning to communicate with deep multi-agent reinforcement learning
Jakob Foerster, Ioannis Alexandros Assael, Nando De Freitas, and Shimon Whiteson · 2016
Cited alongside, same era.
Cryptonets: Applying neural networks to encrypted data with high throughput and accuracy
Ran Gilad-Bachrach, Nathan Dowlin, Kim Laine, Kristin Lauter, Michael Naehrig, and John Wernsing · 2016
Cited alongside, same era.
Deep residual learning for image recognition
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun · 2016
Cited alongside, same era.
Oblivious multi-party machine learning on trusted processors
Olga Ohrimenko, Felix Schuster, Cédric Fournet, Aastha Mehta, Sebastian Nowozin, Kapil Vaswani, and Manuel Costa · 2016
Cited alongside, same era.
Secure multiparty computation from sgx
Raad Bahmani, Manuel Barbosa, Ferdinand Brasser, Bernardo Portela, Ahmad-Reza Sadeghi, Guillaume Scerri, and Bogdan Warinschi · 2017
Cited alongside, same era.
Practical secure aggregation for privacy-preserving machine learning
Keith Bonawitz, Vladimir Ivanov, Ben Kreuter, Antonio Marcedone, H Brendan McMahan, Sarvar Patel, Daniel Ramage, Aaron Segal, and Karn Seth · 2017
Cited alongside, same era.
Privacy-preserving distributed linear regression on high-dimensional data
Adrià Gascón, Phillipp Schoppmann, Borja Balle, Mariana Raykova, Jack Doerner, Samee Zahur, and David Evans · 2017
Cited alongside, same era.
Deep learning for finance: deep portfolios
JB Heaton, NG Polson, and Jan Hendrik Witte · 2017
Cited alongside, same era.
Mobilenets: Efficient convolutional neural networks for mobile vision applications
Original
Andrew G Howard, Menglong Zhu, Bo Chen, Dmitry Kalenichenko, Weijun Wang, Tobias Weyand, Marco Andreetto, and Hartwig Adam · 2017
Cited alongside, same era.
Oblivious neural network predictions via minionn transformations
Jian Liu, Mika Juuti, Yao Lu, and Nadarajah Asokan · 2017
Cited alongside, same era.
Secureml: A system for scalable privacy-preserving machine learning
Payman Mohassel and Yupeng Zhang · 2017
Cited alongside, same era.

Then

Rethinking floating point for deep learning
Original
Jeff Johnson · 2018
Later among the works it cites.
{ \{ GAZELLE } \} : A low latency framework for secure neural network inference
Chiraag Juvekar, Vinod Vaikuntanathan, and Anantha Chandrakasan · 2018
Later among the works it cites.
Privacy aware offloading of deep neural networks
Original
Sam Leroux, Tim Verbelen, Pieter Simoens, and Bart Dhoedt · 2018
Later among the works it cites.
Aby3: A mixed protocol framework for machine learning
Payman Mohassel and Peter Rindal · 2018
Later among the works it cites.
Intel unveils Nervana Neural Net L-1000 for accelerated AI training , 2018
Intel Nervana · 2018
Later among the works it cites.
Mobilenetv2: Inverted residuals and linear bottlenecks
Mark Sandler, Andrew Howard, Menglong Zhu, Andrey Zhmoginov, and Liang-Chieh Chen · 2018
Later among the works it cites.
Slalom: Fast, verifiable and private execution of neural networks in trusted hardware
Original
Florian Tramer and Dan Boneh · 2018
Later among the works it cites.
Not just privacy: Improving performance of private deep learning in mobile cloud
Ji Wang, Jianguo Zhang, Weidong Bao, Xiaomin Zhu, Bokai Cao, and Philip S Yu · 2018
Later among the works it cites.
A guide to deep learning in healthcare
Andre Esteva, Alexandre Robicquet, Bharath Ramsundar, Volodymyr Kuleshov, Mark DePristo, Katherine Chou, Claire Cui, Greg Corrado, Sebastian Thrun, and Jeff Dean · 2019
Later among the works it cites.
Quadd: Quantifying accelerator disaggregated datacenter efficiency
Anubhav Guleria, J Lakshmi, and Chakri Padala · 2019
Later among the works it cites.
Floatpim: In-memory acceleration of deep neural network training with high precision
Mohsen Imani, Saransh Gupta, Yeseong Kim, and Tajana Rosing · 2019
Later among the works it cites.
A study of bfloat16 for deep learning training
Original
Dhiraj Kalamkar, Dheevatsa Mudigere, Naveen Mellempudi, Dipankar Das, Kunal Banerjee, Sasikanth Avancha, Dharma Teja Vooturi, Nataraj Jammalamadaka, Jianyu Huang, Hector Yuen, et al · 2019
Later among the works it cites.
Deep learning on private data
M Sadegh Riazi, Bita Darvish Rouani, and Farinaz Koushanfar · 2019
Later among the works it cites.
Codedprivateml: A fast and privacy-preserving framework for distributed machine learning
Original
Jinhyun So, Basak Guler, A Salman Avestimehr, and Payman Mohassel · 2019
Later among the works it cites.
Securenn: 3-party secure computation for neural network training
Sameer Wagh, Divya Gupta, and Nishanth Chandran · 2019
Later among the works it cites.
Deep leakage from gradients
Ligeng Zhu, Zhijian Liu, and Song Han · 2019
Later among the works it cites.
Machine Learning on AWS , 2020
Amazon · 2020
Closest in time.
Google AI platform , 2020
Google · 2020
Closest in time.
Secure multiparty computations in floating-point arithmetic
Original
Chuan Guo, Awni Hannun, Brian Knott, Laurens van der Maaten, Mark Tygert, and Ruiyu Zhu · 2020
Closest in time.
Azure Machine Learning , 2020
Microsoft · 2020
Closest in time.
Shredder: Learning noise distributions to protect inference privacy
Fatemehsadat Mireshghallah, Mohammadkazem Taram, Prakash Ramrakhyani, Ali Jalali, Dean Tullsen, and Hadi Esmaeilzadeh · 2020
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…