Research interests in the robustness of deep neural networks against domain shifts have been rapidly increasing in recent years. Most existing works, however, focus on improving the accuracy of the model, not the calibration performance which is another important requirement for trustworthy AI systems. Temperature scaling (TS), an accuracy-preserving post-hoc calibration method, has been proven to be effective in in-domain settings, but not in out-of-domain (OOD) due to the difficulty in obtaining a validation set for the unseen domain beforehand. In this paper, we propose consistency-guided temperature scaling (CTS), a new temperature scaling strategy that can significantly enhance the OOD calibration performance by providing mutual supervision among data samples in the source domains. Motivated by our observation that over-confidence stemming from inconsistent sample predictions is the main obstacle to OOD calibration, we propose to guide the scaling process by taking consistencies into account in terms of two different aspects -- style and content -- which are the key components that can well-represent data samples in multi-domain settings. Experimental results demonstrate that our proposed strategy outperforms existing works, achieving superior OOD calibration performance on various datasets. This can be accomplished by employing only the source domains without compromising accuracy, making our scheme directly applicable to various trustworthy AI systems.

翻译：近年来，深度神经网络对域偏移鲁棒性的研究兴趣迅速增长。然而，现有工作大多聚焦于提升模型精度，而忽视了校准性能——这一可信人工智能系统的另一重要需求。温度缩放（TS）作为一种保持精度的后处理校准方法，已被证明在域内场景中有效，但在域外（OOD）场景中因无法预先获取未见域的验证集而效果受限。本文提出一致性引导的温度缩放（CTS），一种通过源域数据样本间相互监督显著提升OOD校准性能的新策略。受"源于样本预测不一致的过度自信是OOD校准主要障碍"这一观察启发，我们提出从风格与内容两个关键维度——即多域场景中充分表征数据样本的核心要素——将一致性纳入缩放过程。实验结果表明，所提策略优于现有方法，在多个数据集上实现了卓越的OOD校准性能。该方案仅需使用源域即可实现，且不牺牲精度，可直接应用于各类可信人工智能系统。