We propose a framework for learning calibrated uncertainties under domain shifts, where the source (training) distribution differs from the target (test) distribution. We detect such domain shifts via a differentiable density ratio estimator and train it together with the task network, composing an adjusted softmax predictive form concerning domain shift. In particular, the density ratio estimation reflects the closeness of a target (test) sample to the source (training) distribution. We employ it to adjust the uncertainty of prediction in the task network. This idea of using the density ratio is based on the distributionally robust learning (DRL) framework, which accounts for the domain shift by adversarial risk minimization. We show that our proposed method generates calibrated uncertainties that benefit downstream tasks, such as unsupervised domain adaptation (UDA) and semi-supervised learning (SSL). On these tasks, methods like self-training and FixMatch use uncertainties to select confident pseudo-labels for re-training. Our experiments show that the introduction of DRL leads to significant improvements in cross-domain performance. We also show that the estimated density ratios align with human selection frequencies, suggesting a positive correlation with a proxy of human perceived uncertainties.

翻译：我们提出了一种在域漂移情境下学习校准不确定性的框架，其中源（训练）分布与目标（测试）分布存在差异。通过可微密度比估计器检测此类域漂移，并将其与任务网络联合训练，构建出关于域漂移的调整型softmax预测形式。具体而言，密度比估计反映了目标（测试）样本与源（训练）分布之间的接近程度，并用于调整任务网络中的预测不确定性。这一基于密度比的思想源于分布鲁棒学习（DRL）框架，该框架通过对抗性风险最小化来处理域漂移。实验表明，所提方法生成的校准不确定性有利于下游任务，如无监督域自适应（UDA）和半监督学习（SSL）。在这些任务中，自训练和FixMatch等方法利用不确定性选择置信伪标签进行重训练。我们的实验表明，引入DRL可显著提升跨领域性能。同时，估计的密度比与人类选择频率具有一致性，表明其与人类感知不确定性的代理变量呈正相关。