In domain adaptation, covariate shift and label shift problems are two distinct and complementary tasks. In covariate shift adaptation where the differences in data distribution arise from variations in feature probabilities, existing approaches naturally address this problem based on \textit{feature probability matching} (\textit{FPM}). However, for label shift adaptation where the differences in data distribution stem solely from variations in class probability, current methods still use FPM on the $d$-dimensional feature space to estimate the class probability ratio on the one-dimensional label space. To address label shift adaptation more naturally and effectively, inspired by a new representation of the source domain's class probability, we propose a new framework called \textit{class probability matching} (\textit{CPM}) which matches two class probability functions on the one-dimensional label space to estimate the class probability ratio, fundamentally different from FPM operating on the $d$-dimensional feature space. Furthermore, by incorporating the kernel logistic regression into the CPM framework to estimate the conditional probability, we propose an algorithm called \textit{class probability matching using kernel methods} (\textit{CPMKM}) for label shift adaptation. From the theoretical perspective, we establish the optimal convergence rates of CPMKM with respect to the cross-entropy loss for multi-class label shift adaptation. From the experimental perspective, comparisons on real datasets demonstrate that CPMKM outperforms existing FPM-based and maximum-likelihood-based algorithms.
翻译:在领域自适应中,协变量偏移和标签偏移问题是两个不同且互补的任务。在协变量偏移适应中,数据分布的差异源于特征概率的变化,现有方法自然通过特征概率匹配(FPM)来解决这一问题。然而,对于标签偏移适应,数据分布的差异仅源于类概率的变化,当前方法仍使用d维特征空间上的FPM来估计一维标签空间上的类概率比。为了更自然且有效地解决标签偏移适应问题,受源域类概率新表示的启发,我们提出了一种名为类概率匹配(CPM)的新框架。该框架通过在一维标签空间上匹配两个类概率函数来估计类概率比,与在d维特征空间上操作的FPM有本质区别。此外,通过将核逻辑回归融入CPM框架以估计条件概率,我们提出了一种基于核方法的类概率匹配算法(CPMKM)用于标签偏移适应。从理论层面,我们建立了CPMKM在多类标签偏移适应下关于交叉熵损失的最优收敛速率。从实验层面,在真实数据集上的比较表明,CPMKM优于现有的基于FPM和基于最大似然的算法。