Two-level overlapping additive Schwarz preconditioner for training scientific machine learning applications

Understand

We introduce a novel two-level overlapping additive Schwarz preconditioner for accelerating the training of scientific machine learning applications.

• The design of the proposed preconditioner is motivated by the nonlinear two-level overlapping additive Schwarz preconditioner.
• The neural network parameters are decomposed into groups (subdomains) with overlapping regions.
• In addition, the network's feed-forward structure is indirectly imposed through a novel subdomain-wise synchronization strategy and a coarse-level training step.