Technical Report for ICCV 2021 Challenge SSLAD-Track3B: Transformers Are Better Continual Learners

Understand

In the SSLAD-Track 3B challenge on continual learning, we propose the method of COntinual Learning with Transformer (COLT).

• We find that transformers suffer less from catastrophic forgetting compared to convolutional neural network.
• The major principle of our method is to equip the transformer based feature extractor with old knowledge distillation and head expanding strategies to compete catastrophic forgetting.
• In this report, we first introduce the overall framework of continual learning for object detection.