Transfer learning (TL) has emerged as a powerful tool to supplement data collected for a target task with data collected for a related source task. The Bayesian framework is natural for TL because information from the source data can be incorporated in the prior distribution for the target data analysis. In this paper, we propose and study Bayesian TL methods for the normal-means problem and multiple linear regression. We propose two classes of prior distributions. The first class assumes the difference in the parameters for the source and target tasks is sparse, i.e., many parameters are shared across tasks. The second assumes that none of the parameters are shared across tasks, but the differences are bounded in $\ell_2$-norm. For the sparse case, we propose a Bayes shrinkage estimator with theoretical guarantees under mild assumptions. The proposed methodology is tested on synthetic data and outperforms state-of-the-art TL methods. We then use this method to fine-tune the last layer of a neural network model to predict the molecular gap property in a material science application. We report improved performance compared to classical fine tuning and methods using only the target data.

翻译：迁移学习（TL）已成为一种强大工具，能够利用相关源任务收集的数据来补充目标任务的数据。贝叶斯框架天然适用于迁移学习，因为源数据的信息可融入目标数据分析的先验分布中。本文针对正态均值问题和多元线性回归，提出并研究了贝叶斯迁移学习方法。我们构建了两类先验分布：第一类假设源任务与目标任务的参数差异具有稀疏性，即大量参数在任务间共享；第二类假设所有参数均不共享，但参数差异受$\ell_2$范数约束。针对稀疏情形，我们提出了一种贝叶斯收缩估计器，并在温和假设下给出理论保证。该方法在合成数据上的测试表现优于现有最优迁移学习方法。进一步地，我们将该方法应用于材料科学中分子带隙性质的预测任务，通过微调神经网络模型的最后一层，相较于经典微调方法和仅使用目标数据的方法，取得了更优的性能提升。