PolyNet: A Pursuit of Structural Diversity in Very Deep Networks

Understand

A number of studies have shown that increasing the depth or width of convolutional networks is a rewarding approach to improve the performance of image recognition.

• In our study, however, we observed difficulties along both directions.
• On one hand, the pursuit for very deep networks is met with a diminishing return and increased training difficulty; on the other hand, widening a network would result in a quadratic growth in both computational cost and memory demand.
• These difficulties motivate us to explore structural diversity in designing deep networks, a new dimension beyond just depth and width.