Tensor Regression Networks with various Low-Rank Tensor Approximations

Understand

Tensor regression networks achieve high compression rate of neural networks while having slight impact on performances.

• They do so by imposing low tensor rank structure on the weight matrices of fully connected layers.
• In recent years, tensor regression networks have been investigated from the perspective of their compressive power, however, the regularization effect of enforcing low-rank tensor structure has not been investigated enough.
• We study tensor regression networks using various low-rank tensor approximations, aiming to compare the compressive and regularization power of different low-rank constraints.