Spatial Sharing of GPU for Autotuning DNN models

Understand

GPUs are used for training, inference, and tuning the machine learning models.

• However, Deep Neural Network (DNN) vary widely in their ability to exploit the full power of high-performance GPUs.
• Spatial sharing of GPU enables multiplexing several DNNs on the GPU and can improve GPU utilization, thus improving throughput and lowering latency.
• DNN models given just the right amount of GPU resources can still provide low inference latency, just as much as dedicating all of the GPU for their inference task.