We develop and analyze a principled approach to kernel ridge regression under covariate shift. The goal is to learn a regression function with small mean squared error over a target distribution, based on unlabeled data from there and labeled data that may have a different feature distribution. We propose to split the labeled data into two subsets and conduct kernel ridge regression on them separately to obtain a collection of candidate models and an imputation model. We use the latter to fill the missing labels and then select the best candidate model accordingly. Our non-asymptotic excess risk bounds show that in quite general scenarios, our estimator adapts to the structure of the target distribution as well as the covariate shift. It achieves the minimax optimal error rate up to a logarithmic factor. The use of pseudo-labels in model selection does not have major negative impacts.

翻译：本文提出并分析了一种在协变量偏移下进行核岭回归的原则性方法。其目标是在仅拥有目标分布的无标签数据以及可能具有不同特征分布的有标签数据时，学习一个在目标分布上均方误差较小的回归函数。我们建议将有标签数据分为两个子集，分别进行核岭回归，以获得候选模型集合和一个插补模型。利用后者填补缺失标签，并据此选择最佳候选模型。我们的非渐近超额风险界限表明，在相当一般的场景下，我们的估计量能够适应目标分布的结构以及协变量偏移，其误差率在忽略对数因子的情况下达到极小极大最优。使用伪标签进行模型选择不会产生显著的负面影响。