Transfer learning is known to perform efficiently in many applications empirically, yet limited literature reports the mechanism behind the scene. This study establishes both formal derivations and heuristic analysis to formulate the theory of transfer learning in deep learning. Our framework utilizing layer variational analysis proves that the success of transfer learning can be guaranteed with corresponding data conditions. Moreover, our theoretical calculation yields intuitive interpretations towards the knowledge transfer process. Subsequently, an alternative method for network-based transfer learning is derived. The method shows an increase in efficiency and accuracy for domain adaptation. It is particularly advantageous when new domain data is sufficiently sparse during adaptation. Numerical experiments over diverse tasks validated our theory and verified that our analytic expression achieved better performance in domain adaptation than the gradient descent method.

翻译：迁移学习在许多实际应用中已被证明能够高效执行，但揭示其背后机理的文献仍较为有限。本研究通过形式化推导与启发式分析相结合的方法，系统构建了深度学习迁移学习的理论框架。我们提出的基于层级变分分析的理论框架证明，在相应数据条件下迁移学习的成功具有理论保证。此外，理论计算为知识迁移过程提供了直观解释，并由此推导出网络迁移学习的替代方法。该方法在领域适应任务中展现出更高的效率与准确性，尤其当新领域数据在适应过程中极度稀疏时具有显著优势。跨多个任务的数值实验验证了所提理论，并证明我们的解析表达式在领域适应中性能优于梯度下降方法。