Learning Time/Memory-Efficient Deep Architectures with Budgeted Super Networks

Understand

We propose to focus on the problem of discovering neural network architectures efficient in terms of both prediction quality and cost.

• For instance, our approach is able to solve the following tasks: learn a neural network able to predict well in less than 100 milliseconds or learn an efficient model that fits in a 50 Mb memory.
• Our contribution is a novel family of models called Budgeted Super Networks (BSN).
• They are learned using gradient descent techniques applied on a budgeted learning objective function which integrates a maximum authorized cost, while making no assumption on the nature of this cost.