How to estimate the uncertainty of a given model is a crucial problem. Current calibration techniques treat different classes equally and thus implicitly assume that the distribution of training data is balanced, but ignore the fact that real-world data often follows a long-tailed distribution. In this paper, we explore the problem of calibrating the model trained from a long-tailed distribution. Due to the difference between the imbalanced training distribution and balanced test distribution, existing calibration methods such as temperature scaling can not generalize well to this problem. Specific calibration methods for domain adaptation are also not applicable because they rely on unlabeled target domain instances which are not available. Models trained from a long-tailed distribution tend to be more overconfident to head classes. To this end, we propose a novel knowledge-transferring-based calibration method by estimating the importance weights for samples of tail classes to realize long-tailed calibration. Our method models the distribution of each class as a Gaussian distribution and views the source statistics of head classes as a prior to calibrate the target distributions of tail classes. We adaptively transfer knowledge from head classes to get the target probability density of tail classes. The importance weight is estimated by the ratio of the target probability density over the source probability density. Extensive experiments on CIFAR-10-LT, MNIST-LT, CIFAR-100-LT, and ImageNet-LT datasets demonstrate the effectiveness of our method.

翻译：如何估计给定模型的不确定性是一个关键问题。现有校准技术平等对待不同类别，因此隐含假设训练数据分布是平衡的，但忽略了现实世界数据常遵循长尾分布这一事实。本文探索了在长尾分布下训练的模型的校准问题。由于不平衡的训练分布与平衡的测试分布之间存在差异，现有校准方法（如温度缩放）难以泛化到该问题。针对领域适配的特定校准方法也不适用，因为它们依赖于不可用的未标记目标域实例。从长尾分布训练的模型往往对头部类别过度自信。为此，我们提出了一种基于知识迁移的校准方法，通过估计尾部类别样本的重要性权重来实现长尾校准。该方法将每个类别的分布建模为高斯分布，并将头部类别的源统计量视为先验，以校准尾部类别的目标分布。我们自适应地将知识从头部类别迁移，以获取尾部类别的目标概率密度。重要性权重通过目标概率密度与源概率密度的比值来估计。在CIFAR-10-LT、MNIST-LT、CIFAR-100-LT和ImageNet-LT数据集上的大量实验证明了我们方法的有效性。