Quiver: Supporting GPUs for Low-Latency, High-Throughput GNN Serving with Workload Awareness

Understand

Systems for serving inference requests on graph neural networks (GNN) must combine low latency with high throughout, but they face irregular computation due to skew in the number of sampled graph nodes and aggregated GNN features.

• This makes it challenging to exploit GPUs effectively: using GPUs to sample only a few graph nodes yields lower performance than CPU-based sampling; and aggregating many features exhibits high data movement costs between GPUs and CPUs.
• Therefore, current GNN serving systems use CPUs for graph sampling and feature aggregation, limiting throughput.
• We describe Quiver, a distributed GPU-based GNN serving system with low-latency and high-throughput.