RNNPool: Efficient Non-linear Pooling for RAM Constrained Inference

Understand

Standard Convolutional Neural Networks (CNNs) designed for computer vision tasks tend to have large intermediate activation maps.

• These require large working memory and are thus unsuitable for deployment on resource-constrained devices typically used for inference on the edge.
• Aggressively downsampling the images via pooling or strided convolutions can address the problem but leads to a significant decrease in accuracy due to gross aggregation of the feature map by standard pooling operators.
• In this paper, we introduce RNNPool, a novel pooling operator based on Recurrent Neural Networks (RNNs), that efficiently aggregates features over large patches of an image and rapidly downsamples activation maps.