Rethinking Pre-training and Self-training

Understand

Pre-training is a dominant paradigm in computer vision.

• For example, supervised ImageNet pre-training is commonly used to initialize the backbones of object detection and segmentation models.
• He et al., however, show a surprising result that ImageNet pre-training has limited impact on COCO object detection.
• Here we investigate self-training as another method to utilize additional data on the same setup and contrast it against ImageNet pre-training.