Supervised transfer learning has received considerable attention due to its potential to boost the predictive power of machine learning in scenarios where data are scarce. Generally, a given set of source models and a dataset from a target domain are used to adapt the pre-trained models to a target domain by statistically learning domain shift and domain-specific factors. While such procedurally and intuitively plausible methods have achieved great success in a wide range of real-world applications, the lack of a theoretical basis hinders further methodological development. This paper presents a general class of transfer learning regression called affine model transfer, following the principle of expected-square loss minimization. It is shown that the affine model transfer broadly encompasses various existing methods, including the most common procedure based on neural feature extractors. Furthermore, the current paper clarifies theoretical properties of the affine model transfer such as generalization error and excess risk. Through several case studies, we demonstrate the practical benefits of modeling and estimating inter-domain commonality and domain-specific factors separately with the affine-type transfer models.

翻译：监督式迁移学习因其在数据稀缺场景下提升机器学习预测能力的潜力而受到广泛关注。通常，通过统计学习领域偏移和领域特定因素，利用一组给定的源模型和目标域数据集，将预训练模型适配到目标域。虽然这类在程序和直觉上合理的方法已在众多实际应用中取得了巨大成功，但理论基础的缺乏阻碍了方法的进一步发展。本文提出一类通用的迁移学习回归方法，称为仿射模型迁移，遵循期望平方损失最小化原则。研究表明，仿射模型迁移广泛涵盖了包括基于神经特征提取器的通用方法在内的多种现有技术。此外，本文阐明了仿射模型迁移的理论性质，如泛化误差和过量风险。通过多个案例研究，我们展示了使用仿射型迁移模型分别建模和估计跨领域共性与领域特定因素的实际优势。