Disentanglement with Biological Constraints: A Theory of Functional Cell Types

Understand

Neurons in the brain are often finely tuned for specific task variables.

• Moreover, such disentangled representations are highly sought after in machine learning.
• Here we mathematically prove that simple biological constraints on neurons, namely nonnegativity and energy efficiency in both activity and weights, promote such sought after disentangled representations by enforcing neurons to become selective for single factors of task variation.
• We demonstrate these constraints lead to disentanglement in a variety of tasks and architectures, including variational autoencoders.