Transfer learning is known to perform efficiently in many applications empirically, yet limited literature reports the mechanism behind the scene. This study establishes both formal derivations and heuristic analysis to formulate the theory of transfer learning in deep learning. Our framework utilizing layer variational analysis proves that the success of transfer learning can be guaranteed with corresponding data conditions. Moreover, our theoretical calculation yields intuitive interpretations towards the knowledge transfer process. Subsequently, an alternative method for network-based transfer learning is derived. The method shows an increase in efficiency and accuracy for domain adaptation. It is particularly advantageous when new domain data is sufficiently sparse during adaptation. Numerical experiments over diverse tasks validated our theory and verified that our analytic expression achieved better performance in domain adaptation than the gradient descent method.

翻译：迁移学习在众多应用中被实证证明能够高效执行，但揭示其内在机制的文献仍十分有限。本研究通过形式化推导与启发式分析，构建了深度学习迁移学习的理论框架。我们提出的基于层级变分分析的理论框架证明，在相应的数据条件下，迁移学习的成功具有理论保证。此外，我们的理论计算为知识迁移过程提供了直观的解释。基于此，我们进一步推导出一种替代性的网络迁移学习方法，该方法在域适应任务中展现出更高的效率与准确性，尤其在新域数据稀疏的适应场景下优势显著。跨多种任务的数值实验验证了我们的理论，并证实与梯度下降法相比，我们的解析表达式在域适应中实现了更优的性能。