Deconstructing Data Reconstruction: Multiclass, Weight Decay and General Losses

Understand

Memorization of training data is an active research area, yet our understanding of the inner workings of neural networks is still in its infancy.

• Recently, Haim et al.
• (2022) proposed a scheme to reconstruct training samples from multilayer perceptron binary classifiers, effectively demonstrating that a large portion of training samples are encoded in the parameters of such networks.
• In this work, we extend their findings in several directions, including reconstruction from multiclass and convolutional neural networks.