Transfer learning aims to facilitate the learning of a target domain by transferring knowledge from a source domain. The source domain typically contains semantically meaningful samples (*e.g.*, images) to facilitate effective knowledge transfer. However, a recent study observes that the noise domain constructed from simple distributions (*e.g.*, Gaussian distributions) can serve as a surrogate source domain in the semi-supervised setting, where only a small proportion of target samples are labeled while most remain unlabeled. Based on this surprising observation, we formulate a novel problem termed *Semi-Supervised Noise Adaptation* (SSNA), which aims to leverage a synthetic noise domain to improve the generalization of the target domain. To address this problem, we first establish a generalization bound characterizing the effect of the noise domain on generalization, based on which we propose a Noise Adaptation Framework (NAF). Extensive experiments demonstrate that NAF effectively leverages the noise domain to tighten the generalization bound of the target domain, leading to improved performance. The codes are available at https://github.com/AIResearch-Group/SSNA.
翻译:迁移学习旨在通过从源域迁移知识来促进目标域的学习。源域通常包含语义上有意义的样本(例如图像),以促进有效的知识迁移。然而,近期一项研究发现,由简单分布(例如高斯分布)构建的噪声域可以在半监督设置中作为替代源域,该设置中仅有一小部分目标样本有标签,而大部分样本无标签。基于这一惊人发现,我们定义了一个称为*半监督噪声适应*(SSNA)的新问题,旨在利用合成噪声域来提升目标域的泛化能力。为解决该问题,我们首先建立了一个表征噪声域对泛化影响的泛化界,并在此基础上提出了一个噪声适应框架(NAF)。大量实验表明,NAF有效利用噪声域来收紧目标域的泛化界,从而提升了性能。相关代码可在https://github.com/AIResearch-Group/SSNA获取。