Collaborative Distillation for Ultra-Resolution Universal Style Transfer

Understand

Universal style transfer methods typically leverage rich representations from deep Convolutional Neural Network (CNN) models (e.g., VGG-19) pre-trained on large collections of images.

• Despite the effectiveness, its application is heavily constrained by the large model size to handle ultra-resolution images given limited memory.
• In this work, we present a new knowledge distillation method (named Collaborative Distillation) for encoder-decoder based neural style transfer to reduce the convolutional filters.
• The main idea is underpinned by a finding that the encoder-decoder pairs construct an exclusive collaborative relationship, which is regarded as a new kind of knowledge for style transfer models.