Diversity Networks: Neural Network Compression Using Determinantal Point Processes

Understand

We introduce Divnet, a flexible technique for learning networks with diverse neurons.

• Divnet models neuronal diversity by placing a Determinantal Point Process (DPP) over neurons in a given layer.
• It uses this DPP to select a subset of diverse neurons and subsequently fuses the redundant neurons into the selected ones.
• Compared with previous approaches, Divnet offers a more principled, flexible technique for capturing neuronal diversity and thus implicitly enforcing regularization.