2020

Quantifying Privacy Leakage in Graph Embedding

Duddu, Vasisht, Boutet, Antoine, Shejwalkar, Virat

Understand

Graph embeddings have been proposed to map graph data to low dimensional space for downstream processing (e.g., node classification or link prediction).

With the increasing collection of personal data, graph embeddings can be trained on private and sensitive data.
For the first time, we quantify the privacy leakage in graph embeddings through three inference attacks targeting Graph Neural Networks.
We propose a membership inference attack to infer whether a graph node corresponding to individual user's data was member of the model's training or not.

Built on

Information Leakage in Embedding Models
Original
Congzheng Song and Ananth Raghunathan. 2020 · 2004
Earlier work this paper cites.
Visualizing Data using t-SNE
Laurens van der Maaten and Geoffrey Hinton. 2008 · 2008
Earlier work this paper cites.
Inferring interests from mobility and social interactions. In
Anastasios Noulas, Mirco Musolesi, Massimiliano Pontil, and Cecilia Mascolo. 2009 · 2009
Earlier work this paper cites.
Distributed Representations of Words and Phrases and Their Compositionality. In
Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013 · 2013
Earlier work this paper cites.
DeepWalk: Online Learning of Social Representations. In
Bryan Perozzi, Rami Al-Rfou, and Steven Skiena. 2014 · 2014
Earlier work this paper cites.
Hacking Smart Machines with Smarter Ones: How to Extract Meaningful Data from Machine Learning Classifiers
Giuseppe Ateniese, Luigi V. Mancini, Angelo Spognardi, Antonio Villani, Domenico Vitali, and Giovanni Felici. 2015 · 2015
Earlier work this paper cites.
Model Inversion Attacks That Exploit Confidence Information and Basic Countermeasures. In
Matt Fredrikson, Somesh Jha, and Thomas Ristenpart. 2015 · 2015
Earlier work this paper cites.
Deep Learning with Differential Privacy. In
Martin Abadi, Andy Chu, Ian Goodfellow, H. Brendan McMahan, Ilya Mironov, Kunal Talwar, and Li Zhang. 2016 · 2016
Earlier work this paper cites.
node2vec: Scalable Feature Learning for Networks. In
Aditya Grover and Jure Leskovec. 2016 · 2016
Earlier work this paper cites.
Inductive Representation Learning on Large Graphs
Will Hamilton, Zhitao Ying, and Jure Leskovec. 2017 · 2017
Earlier work this paper cites.
AttriInfer: Inferring User Attributes in Online Social Networks Using Markov Random Fields. In
Jinyuan Jia, Binghui Wang, Le Zhang, and Neil Zhenqiang Gong. 2017 · 2017
Earlier work this paper cites.
Semi-Supervised Classification with Graph Convolutional Networks. In
Thomas N. Kipf and Max Welling. 2017 · 2017
Earlier work this paper cites.
Membership Inference Attacks Against Machine Learning Models. In
R. Shokri, M. Stronati, C. Song, and V. Shmatikov. 2017 · 2017
Earlier work this paper cites.
Machine Learning Models That Remember Too Much. In
Congzheng Song, Thomas Ristenpart, and Vitaly Shmatikov. 2017 · 2017
Earlier work this paper cites.
A Comprehensive Survey of Graph Embedding: Problems, Techniques, and Applications
H. Cai, V. W. Zheng, and K. C. Chang. 2018 · 2018
Earlier work this paper cites.
A Tutorial on Network Embeddings
Original
Haochen Chen, Bryan Perozzi, Rami Al-Rfou, and Steven Skiena. 2018 · 2018
Earlier work this paper cites.

Similar

Topology Adaptive Graph Convolutional Networks
Original
Jian Du, Shanghang Zhang, Guanhang Wu, José M. F. Moura, and Soummya Kar. 2018 · 2018
Cited alongside, same era.
Stealing Neural Networks via Timing Side Channels
Original
Vasisht Duddu, Debasis Samanta, D Vijay Rao, and Valentina E. Balas. 2018 · 2018
Cited alongside, same era.
Property Inference Attacks on Fully Connected Neural Networks Using Permutation Invariant Representations. In
Karan Ganju, Qi Wang, Wei Yang, Carl A. Gunter, and Nikita Borisov. 2018 · 2018
Cited alongside, same era.
Attriguard: A Practical Defense against Attribute Inference Attacks via Adversarial Machine Learning. In
Jinyuan Jia and Neil Zhenqiang Gong. 2018 · 2018
Cited alongside, same era.
Deeper Insights into Graph Convolutional Networks for Semi-Supervised Learning. In
Qimai Li, Zhichao Han, and Xiao-Ming Wu. 2018 · 2018
Cited alongside, same era.
Machine Learning with Membership Privacy Using Adversarial Regularization. In
Milad Nasr, Reza Shokri, and Amir Houmansadr. 2018 · 2018
Cited alongside, same era.
Graph Attention Networks. In
Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. 2018 · 2018
Cited alongside, same era.
Stealing Hyperparameters in Machine Learning. In
B. Wang and N. Z. Gong. 2018 · 2018
Cited alongside, same era.
DPNE: Differentially Private Network Embedding. 235–246
Depeng Xu, Shuhan Yuan, Xintao Wu, and HaiNhat Phan. 2018 · 2018
Cited alongside, same era.
Graph neural networks: A review of methods and applications
Original
Jie Zhou, Ganqu Cui, Zhengyan Zhang, Cheng Yang, Zhiyuan Liu, Lifeng Wang, Changcheng Li, and Maosong Sun. 2018 · 2018
Cited alongside, same era.
Adversarial Attacks on Neural Networks for Graph Data. In
Daniel Zügner, Amir Akbarnejad, and Stephan Günnemann. 2018 · 2018
Cited alongside, same era.
Adversarial Attacks on Node Embeddings via Graph Poisoning. In
Aleksandar Bojchevski and Stephan Günnemann. 2019 · 2019
Cited alongside, same era.

Then

MemGuard: Defending against Black-Box Membership Inference Attacks via Adversarial Examples. In
Jinyuan Jia, Ahmed Salem, Michael Backes, Yang Zhang, and Neil Zhenqiang Gong. 2019 · 2019
Later among the works it cites.
Combining Neural Networks with Personalized PageRank for Classification on Graphs. In
Johannes Klicpera, Aleksandar Bojchevski, and Stephan Günnemann. 2019 · 2019
Later among the works it cites.
Exploiting Unintended Feature Leakage in Collaborative Learning. In
Luca Melis, Congzheng Song, Emiliano De Cristofaro, and Vitaly Shmatikov. 2019 · 2019
Later among the works it cites.
Comprehensive Privacy Analysis of Deep Learning: Passive and Active White-box Inference Attacks against Centralized and Federated Learning. In
M. Nasr, R. Shokri, and A. Houmansadr. 2019 · 2019
Later among the works it cites.
ML-Leaks: Model and Data Independent Membership Inference Attacks and Defenses on Machine Learning Models. In
Ahmed Salem, Yang Zhang, Mathias Humbert, Pascal Berrang, Mario Fritz, and Michael Backes. 2019 · 2019
Later among the works it cites.
dpUGC: Learn Differentially Private Representation for User Generated Contents. In
Lili Jiang Xuan-Son Vu, Son N. Tran. 2019 · 2019
Later among the works it cites.
Revisiting user mobility and social relationships in lbsns: a hypergraph embedding approach. In
Dingqi Yang, Bingqing Qu, Jie Yang, and Philippe Cudre-Mauroux. 2019 · 2019
Later among the works it cites.
Robust Graph Convolutional Networks Against Adversarial Attacks. In
Dingyuan Zhu, Ziwei Zhang, Peng Cui, and Wenwu Zhu. 2019 · 2019
Later among the works it cites.
Certifiable Robustness and Robust Training for Graph Convolutional Networks. In
Daniel Zügner and Stephan Günnemann. 2019 · 2019
Later among the works it cites.
Privacy Risks of General-Purpose Language Models. In
X. Pan, M. Zhang, S. Ji, and M. Yang. 2020 · 2020
Closest in time.
Overlearning Reveals Sensitive Attributes. In
Congzheng Song and Vitaly Shmatikov. 2020 · 2020
Closest in time.
Systematic Evaluation of Privacy Risks of Machine Learning Models
Original
Liwei Song and Prateek Mittal. 2020 · 2020
Closest in time.
DYSAN: Dynamically sanitizing motion sensor data against sensitive inferences through adversarial networks. In
Antoine Boutet, Carole Frindel, Sébastien Gambs, Théo Jourdan, and Claude Rosin Ngueveu. 2021 · 2021
Closest in time.
Stealing Links from Graph Neural Networks. In
Xinlei He, Jinyuan Jia, Michael Backes, Neil Zhenqiang Gong, and Yang Zhang. 2021 · 2021
Closest in time.
Membership Privacy for Machine Learning Models Through Knowledge Transfer. In
Virat Shejwalkar and Amir Houmansadr. 2021 · 2021
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…