Rare categories abound in a number of real-world networks and play a pivotal role in a variety of high-stakes applications, including financial fraud detection, network intrusion detection, and rare disease diagnosis. Rare category analysis (RCA) refers to the task of detecting, characterizing, and comprehending the behaviors of minority classes in a highly-imbalanced data distribution. While the vast majority of existing work on RCA has focused on improving the prediction performance, a few fundamental research questions heretofore have received little attention and are less explored: How confident or uncertain is a prediction model in rare category analysis? How can we quantify the uncertainty in the learning process and enable reliable rare category analysis? To answer these questions, we start by investigating miscalibration in existing RCA methods. Empirical results reveal that state-of-the-art RCA methods are mainly over-confident in predicting minority classes and under-confident in predicting majority classes. Motivated by the observation, we propose a novel individual calibration framework, named CALIRARE, for alleviating the unique challenges of RCA, thus enabling reliable rare category analysis. In particular, to quantify the uncertainties in RCA, we develop a node-level uncertainty quantification algorithm to model the overlapping support regions with high uncertainty; to handle the rarity of minority classes in miscalibration calculation, we generalize the distribution-based calibration metric to the instance level and propose the first individual calibration measurement on graphs named Expected Individual Calibration Error (EICE). We perform extensive experimental evaluations on real-world datasets, including rare category characterization and model calibration tasks, which demonstrate the significance of our proposed framework.

翻译：真实世界的网络中普遍存在罕见类别，并在金融欺诈检测、网络入侵检测及罕见疾病诊断等高影响力应用中发挥关键作用。罕见类别分析（RCA）指在高度不平衡数据分布中检测、表征并理解少数类行为的过程。尽管现有RCA研究大多聚焦于提升预测性能，但若干基础研究问题迄今未获重视且探索不足：罕见类别分析中预测模型的可信度或不确定性如何度量？如何量化学习过程中的不确定性以实现可靠分析？为回答这些问题，我们首先探究现有RCA方法的校准偏差。实验表明，当前最先进的RCA方法对少数类预测过度自信，而对多数类预测信心不足。基于此发现，我们提出新颖的个体校准框架CALIRARE，以缓解RCA面临的独特挑战，进而实现可靠分析。具体而言，为量化RCA中的不确定性，我们开发了节点级不确定性量化算法，对高不确定性的重叠支持区域建模；为处理校准偏差计算中少数类的稀缺性，我们将基于分布的校准指标泛化至实例级别，并提出首个图个体校准度量——期望个体校准误差（EICE）。我们在真实数据集上开展广泛实验，涵盖罕见类别表征与模型校准任务，证实了所提框架的重要性。