MSCCL++: Rethinking GPU Communication Abstractions for AI Inference

Understand

AI applications increasingly run on fast-evolving, heterogeneous hardware to maximize performance, but general-purpose libraries lag in supporting these features.

• Performance-minded programmers often build custom communication stacks that are fast but error-prone and non-portable.
• This paper introduces MSCCL++, a design methodology for developing high-performance, portable communication kernels.
• It provides (1) a low-level, performance-preserving primitive interface that exposes minimal hardware abstractions while hiding the complexities of synchronization and consistency, (2) a higher-level DSL for application developers to implement workload-specific communication algorithms, and (3) a library of efficient algorithms implementing the standard collective API, enabling adoption by users with minimal expertise.