Multi Layer Neural Networks as Replacement for Pooling Operations

Understand

Pooling operations, which can be calculated at low cost and serve as a linear or nonlinear transfer function for data reduction, are found in almost every modern neural network.

• Countless modern approaches have already tackled replacing the common maximum value selection and mean value operations, not to mention providing a function that allows different functions to be selected through changing parameters.
• Additional neural networks are used to estimate the parameters of these pooling functions.Consequently, pooling layers may require supplementary parameters to increase the complexity of the whole model.
• In this work, we show that one perceptron can already be used effectively as a pooling operation without increasing the complexity of the model.