Cryptanalytic Extraction of Neural Network Models

Understand

We argue that the machine learning problem of model extraction is actually a cryptanalytic problem in disguise, and should be studied as such.

• Given oracle access to a neural network, we introduce a differential attack that can efficiently steal the parameters of the remote model up to floating point precision.
• Our attack relies on the fact that ReLU neural networks are piecewise linear functions, and thus queries at the critical points reveal information about the model parameters.
• We evaluate our attack on multiple neural network models and extract models that are 2^20 times more precise and require 100x fewer queries than prior work.