Transfer learning has become a central paradigm in modern machine learning, yet it suffers from the long-standing problem of negative transfer, where leveraging source representations can harm rather than help performance on the target task. Although empirical remedies have been proposed, there remains little theoretical understanding of how to reliably avoid negative transfer. In this paper, we investigate a simple yet remarkably effective strategy: augmenting frozen, pretrained source-side features with a trainable target-side encoder that adapts target features to capture residual signals overlooked by models pretrained on the source data. We show this residual feature integration strategy is sufficient to provably prevent negative transfer, by establishing theoretical guarantees that it has no worse convergence rate than training from scratch under the informative class of target distributions up to logarithmic factors, and that the convergence rate can transition seamlessly from nonparametric to near-parametric when source representations are informative. To our knowledge, this is the first theoretical work that ensures protection against negative transfer. We carry out extensive numerical experiments across image, text and tabular benchmarks, and empirically verify that the method consistently safeguards performance under distribution shift, label noise, semantic perturbation, and class imbalance. We additionally demonstrate that this residual integration mechanism uniquely supports adapt-time multimodality extension, enabling a pretrained single-cell foundation model to incorporate spatial signals for lymph-node anatomical classification despite the source model being trained without them. Our study thus advances the theory of safe transfer learning, and provides a principled approach that is simple, robust, architecture-agnostic, and broadly applicable.

翻译：迁移学习已成为现代机器学习的核心范式，但其长期受到负迁移问题的困扰——即利用源域表征反而可能损害而非提升目标任务的性能。尽管已有经验性缓解方案被提出，关于如何可靠避免负迁移仍缺乏理论理解。本文研究了一种简单却异常有效的策略：通过可训练的目标端编码器增强冻结的预训练源端特征，使目标特征能够捕获源数据预训练模型所忽略的残差信号。我们证明这种残差特征集成策略足以理论保证地防止负迁移：我们建立了理论保证，表明在信息丰富的目标分布类别下（至多对数因子），该策略的收敛速率不差于从零开始训练；且当源表征具有信息量时，收敛速率可无缝地从非参数过渡到近参数。据我们所知，这是首个能确保抵御负迁移的理论工作。我们在图像、文本和表格基准上进行了大量数值实验，实证验证了该方法在分布偏移、标签噪声、语义扰动和类别不平衡条件下均能持续保障性能。此外，我们证明这种残差集成机制独特地支持适应时多模态扩展，使得预训练的单细胞基础模型能够整合空间信号进行淋巴结解剖分类——尽管源模型训练时未包含此类信号。因此，我们的研究推进了安全迁移学习的理论，并提供了一种原理性方法，该方法简单、鲁棒、架构无关且具有广泛适用性。