WSNet: Compact and Efficient Networks Through Weight Sampling

Understand

We present a new approach and a novel architecture, termed WSNet, for learning compact and efficient deep neural networks.

• Existing approaches conventionally learn full model parameters independently and then compress them via ad hoc processing such as model pruning or filter factorization.
• Alternatively, WSNet proposes learning model parameters by sampling from a compact set of learnable parameters, which naturally enforces {parameter sharing} throughout the learning process.
• We demonstrate that such a novel weight sampling approach (and induced WSNet) promotes both weights and computation sharing favorably.