On The Specialization of Neural Modules

Understand

A number of machine learning models have been proposed with the goal of achieving systematic generalization: the ability to reason about new situations by combining aspects of previous experiences.

• These models leverage compositional architectures which aim to learn specialized modules dedicated to structures in a task that can be composed to solve novel problems with similar structures.
• While the compositionality of these architectures is guaranteed by design, the modules specializing is not.
• Here we theoretically study the ability of network modules to specialize to useful structures in a dataset and achieve systematic generalization.