Model Weight Theft With Just Noise Inputs: The Curious Case of the Petulant Attacker

Understand

This paper explores the scenarios under which an attacker can claim that 'Noise and access to the softmax layer of the model is all you need' to steal the weights of a convolutional neural network whose architecture is already known.

• We were able to achieve 96% test accuracy using the stolen MNIST model and 82% accuracy using the stolen KMNIST model learned using only i.i.d.
• Bernoulli noise inputs.
• We posit that this theft-susceptibility of the weights is indicative of the complexity of the dataset and propose a new metric that captures the same.