Beyond Occam's Razor in System Identification: Double-Descent when Modeling Dynamics

Understand

System identification aims to build models of dynamical systems from data.

• Traditionally, choosing the model requires the designer to balance between two goals of conflicting nature; the model must be rich enough to capture the system dynamics, but not so flexible that it learns spurious random effects from the dataset.
• It is typically observed that the model validation performance follows a U-shaped curve as the model complexity increases.
• Recent developments in machine learning and statistics, however, have observed situations where a "double-descent" curve subsumes this U-shaped model-performance curve.