Position: Optimization in SciML Should Employ the Function Space Geometry

Understand

Scientific machine learning (SciML) is a relatively new field that aims to solve problems from different fields of natural sciences using machine learning tools.

• It is well-documented that the optimizers commonly used in other areas of machine learning perform poorly on many SciML problems.
• We provide an infinite-dimensional view on optimization problems encountered in scientific machine learning and advocate for the paradigm first optimize, then discretize for their solution.
• This amounts to first choosing an appropriate infinite-dimensional algorithm which is then discretized in a second step.