MPI-rical: Data-Driven MPI Distributed Parallelism Assistance with Transformers

Understand

Message Passing Interface (MPI) plays a crucial role in distributed memory parallelization across multiple nodes.

• However, parallelizing MPI code manually, and specifically, performing domain decomposition, is a challenging, error-prone task.
• In this paper, we address this problem by developing MPI-RICAL, a novel data-driven, programming-assistance tool that assists programmers in writing domain decomposition based distributed memory parallelization code.
• Specifically, we train a supervised language model to suggest MPI functions and their proper locations in the code on the fly.