Bayesian Tensorized Neural Networks with Automatic Rank Selection

Understand

Tensor decomposition is an effective approach to compress over-parameterized neural networks and to enable their deployment on resource-constrained hardware platforms.

• However, directly applying tensor compression in the training process is a challenging task due to the difficulty of choosing a proper tensor rank.
• In order to achieve this goal, this paper proposes a Bayesian tensorized neural network.
• Our Bayesian method performs automatic model compression via an adaptive tensor rank determination.