Transfer learning is known to perform efficiently in many applications empirically, yet limited literature reports the mechanism behind the scene. This study establishes both formal derivations and heuristic analysis to formulate the theory of transfer learning in deep learning. Our framework utilizing layer variational analysis proves that the success of transfer learning can be guaranteed with corresponding data conditions. Moreover, our theoretical calculation yields intuitive interpretations towards the knowledge transfer process. Subsequently, an alternative method for network-based transfer learning is derived. The method shows an increase in efficiency and accuracy for domain adaptation. It is particularly advantageous when new domain data is sufficiently sparse during adaptation. Numerical experiments over diverse tasks validated our theory and verified that our analytic expression achieved better performance in domain adaptation than the gradient descent method.

翻译：在许多应用中,人们都知道转让学习效率很高,从经验上看,但有限的文献却报告了幕后机制。本研究既建立了正式的衍生法,也建立了超常分析法,以制定深层学习中转让学习理论。我们的框架利用分层变异分析证明,转让学习的成功可以用相应的数据条件来保证。此外,我们的理论计算对知识转让过程产生了直觉的解释。随后,产生了一种基于网络的转让学习的替代方法。该方法表明在区域适应方面的效率和准确性有所提高。当新的域数据在适应过程中足够少的时候,这一方法特别有利。对不同任务的量化实验证实了我们的理论,并证实我们的分析性表达方式在适应领域方面比梯度下降方法取得更好的表现。