On-Device Domain Generalization

Understand

We present a systematic study of domain generalization (DG) for tiny neural networks.

• This problem is critical to on-device machine learning applications but has been overlooked in the literature where research has been merely focused on large models.
• Tiny neural networks have much fewer parameters and lower complexity and therefore should not be trained the same way as their large counterparts for DG applications.
• By conducting extensive experiments, we find that knowledge distillation (KD), a well-known technique for model compression, is much better for tackling the on-device DG problem than conventional DG methods.