Precision-aware Latency and Energy Balancing on Multi-Accelerator Platforms for DNN Inference

Understand

The need to execute Deep Neural Networks (DNNs) at low latency and low power at the edge has spurred the development of new heterogeneous Systems-on-Chips (SoCs) encapsulating a diverse set of hardware accelerators.

• How to optimally map a DNN onto such multi-accelerator systems is an open problem.
• We propose ODiMO, a hardware-aware tool that performs a fine-grain mapping across different accelerators on-chip, splitting individual layers and executing them in parallel, to reduce inference energy consumption or latency, while taking into account each accelerator's quantization precision to maintain accuracy.
• Pareto-optimal networks in the accuracy vs.