Distilling Object Detectors with Task Adaptive Regularization

Understand

Current state-of-the-art object detectors are at the expense of high computational costs and are hard to deploy to low-end devices.

• Knowledge distillation, which aims at training a smaller student network by transferring knowledge from a larger teacher model, is one of the promising solutions for model miniaturization.
• In this paper, we investigate each module of a typical detector in depth, and propose a general distillation framework that adaptively transfers knowledge from teacher to student according to the task specific priors.
• The intuition is that simply distilling all information from teacher to student is not advisable, instead we should only borrow priors from the teacher model where the student cannot perform well.