At the same time that artificial intelligence (AI) and machine learning are becoming central to human life, their potential harms become more vivid. In the presence of such drawbacks, a critical question to address before using individual predictions for critical decision-making is whether those are reliable. Aligned with recent efforts on data-centric AI, this paper proposes a novel approach, complementary to the existing work on trustworthy AI, to address the reliability question through the lens of data. Specifically, it associates data sets with distrust quantification that specifies their scope of use for individual predictions. It develops novel algorithms for efficient and effective computation of distrust values. The proposed algorithms learn the necessary components of the measures from the data itself and are sublinear, which makes them scalable to very large and multi-dimensional settings. Furthermore, an estimator is designed to enable no-data access during the query time. Besides theoretical analyses, the algorithms are evaluated experimentally, using multiple real and synthetic data sets and different tasks. The experiment results reflect a consistent correlation between distrust values and model performance. This highlights the necessity of dismissing prediction outcomes for cases with high distrust values, at least for critical decisions.

翻译：在人工智能和机器学习日益成为人类生活核心的同时，其潜在危害也愈发凸显。面对这些缺陷，在将个体预测用于关键决策之前，一个亟待解决的问题是这些预测是否可靠。基于近期以数据为中心的人工智能研究趋势，本文提出了一种新颖的方法，作为对现有可信人工智能研究的补充，通过数据的视角来探讨可靠性问题。具体而言，该方法将数据集与不信任度量结合起来，以明确其用于个体预测的适用范围。本文开发了用于高效计算不信任值的新算法。所提出的算法从数据本身学习度量的必要组成部分，并具有次线性复杂度，从而使其能够扩展到大规模和多维度的场景。此外，还设计了一个估计器，以便在查询时无需访问数据。除了理论分析外，本文还使用多个真实和合成数据集以及不同任务对算法进行了实验评估。实验结果一致显示，不信任值与模型性能之间存在相关性。这强调了对于关键决策而言，至少需要舍弃具有高不信任值的预测结果。