ApproxDBN: Approximate Computing for Discriminative Deep Belief Networks

Understand

Probabilistic generative neural networks are useful for many applications, such as image classification, speech recognition and occlusion removal.

• However, the power budget for hardware implementations of neural networks can be extremely tight.
• To address this challenge we describe a design methodology for using approximate computing methods to implement Approximate Deep Belief Networks (ApproxDBNs) by systematically exploring the use of (1) limited precision of variables; (2) criticality analysis to identify the nodes in the network which can operate with such limited precision while allowing the network to maintain target accuracy levels; and (3) a greedy search methodology with incremental retraining to determine the optimal reduction in precision to enable maximize power savings under user-specified accuracy constraints.
• Experimental results show that significant bit-length reduction can be achieved by our ApproxDBN with constrained accuracy loss.