Generalized Ternary Connect: End-to-End Learning and Compression of Multiplication-Free Deep Neural Networks

Understand

The use of deep neural networks in edge computing devices hinges on the balance between accuracy and complexity of computations.

• Ternary Connect (TC) \cite{lin2015neural} addresses this issue by restricting the parameters to three levels $-1, 0$, and $+1$, thus eliminating multiplications in the forward pass of the network during prediction.
• We propose Generalized Ternary Connect (GTC), which allows an arbitrary number of levels while at the same time eliminating multiplications by restricting the parameters to integer powers of two.
• The primary contribution is that GTC learns the number of levels and their values for each layer, jointly with the weights of the network in an end-to-end fashion.