Transfer Learning of Linear Regression with Multiple Pretrained Models: Benefiting from More Pretrained Models via Overparameterization Debiasing

Understand

We study transfer learning for a linear regression task using several least-squares pretrained models that can be overparameterized.

• We formulate the target learning task as optimization that minimizes squared errors on the target dataset with penalty on the distance of the learned model from the pretrained models.
• We analytically formulate the test error of the learned target model and provide the corresponding empirical evaluations.
• Our results elucidate when using more pretrained models can improve transfer learning.