Towards Reverse-Engineering Black-Box Neural Networks

Understand

Many deployed learned models are black boxes: given input, returns output.

• Internal information about the model, such as the architecture, optimisation procedure, or training data, is not disclosed explicitly as it might contain proprietary information or make the system more vulnerable.
• This work shows that such attributes of neural networks can be exposed from a sequence of queries.
• This has multiple implications.