Automatic heterogeneous quantization of deep neural networks for low-latency inference on the edge for particle detectors

Understand

Although the quest for more accurate solutions is pushing deep learning research towards larger and more complex algorithms, edge devices demand efficient inference and therefore reduction in model size, latency and energy consumption.

• One technique to limit model size is quantization, which implies using fewer bits to represent weights and biases.
• Such an approach usually results in a decline in performance.
• Here, we introduce a method for designing optimally heterogeneously quantized versions of deep neural network models for minimum-energy, high-accuracy, nanosecond inference and fully automated deployment on chip.