From Maxout to Channel-Out: Encoding Information on Sparse Pathways

Understand

Motivated by an important insight from neural science, we propose a new framework for understanding the success of the recently proposed "maxout" networks.

• The framework is based on encoding information on sparse pathways and recognizing the correct pathway at inference time.
• Elaborating further on this insight, we propose a novel deep network architecture, called "channel-out" network, which takes a much better advantage of sparse pathway encoding.
• In channel-out networks, pathways are not only formed a posteriori, but they are also actively selected according to the inference outputs from the lower layers.