Transfer learning is known to perform efficiently in many applications empirically, yet limited literature reports the mechanism behind the scene. This study establishes both formal derivations and heuristic analysis to formulate the theory of transfer learning in deep learning. Our framework utilizing layer variational analysis proves that the success of transfer learning can be guaranteed with corresponding data conditions. Moreover, our theoretical calculation yields intuitive interpretations towards the knowledge transfer process. Subsequently, an alternative method for network-based transfer learning is derived. The method shows an increase in efficiency and accuracy for domain adaptation. It is particularly advantageous when new domain data is sufficiently sparse during adaptation. Numerical experiments over diverse tasks validated our theory and verified that our analytic expression achieved better performance in domain adaptation than the gradient descent method.

翻译：迁移学习在许多实际应用中展现出高效性能，但关于其内在机制的研究报道仍十分有限。本研究通过形式化推导与启发式分析相结合的方式，系统构建了深度学习迁移学习理论框架。我们提出的基于层级变分分析的理论框架证明，在相应数据条件下迁移学习的有效性可得到保证。此外，理论计算得出了对知识迁移过程的直观解释，并据此推导出网络迁移学习的替代方法。该方法在域适应中展现出更高的效率与准确性，尤其在新领域数据极为稀疏时具有显著优势。跨多种任务的数值实验验证了本理论的正确性，并证明我们的解析表达式在域适应中取得了优于梯度下降法的性能表现。