A Scaling Law for Synthetic-to-Real Transfer: How Much Is Your Pre-training Effective?

Understand

Synthetic-to-real transfer learning is a framework in which a synthetically generated dataset is used to pre-train a model to improve its performance on real vision tasks.

• The most significant advantage of using synthetic images is that the ground-truth labels are automatically available, enabling unlimited expansion of the data size without human cost.
• However, synthetic data may have a huge domain gap, in which case increasing the data size does not improve the performance.
• How can we know that? In this study, we derive a simple scaling law that predicts the performance from the amount of pre-training data.