Deep Learning Alternatives of the Kolmogorov Superposition Theorem

Understand

This paper explores alternative formulations of the Kolmogorov Superposition Theorem (KST) as a foundation for neural network design.

• The original KST formulation, while mathematically elegant, presents practical challenges due to its limited insight into the structure of inner and outer functions and the large number of unknown variables it introduces.
• Kolmogorov-Arnold Networks (KANs) leverage KST for function approximation, but they have faced scrutiny due to mixed results compared to traditional multilayer perceptrons (MLPs) and practical limitations imposed by the original KST formulation.
• To address these issues, we introduce ActNet, a scalable deep learning model that builds on the KST and overcomes many of the drawbacks of Kolmogorov's original formulation.