Artificial intelligence (AI) technology and systems have been advancing rapidly. However, ensuring the reliability of these systems is crucial for fostering public confidence in their use. This necessitates the modeling and analysis of reliability data specific to AI systems. A major challenge in AI reliability research, particularly for those in academia, is the lack of readily available AI reliability data. To address this gap, this paper focuses on conducting a comprehensive review of available AI reliability data and establishing DR-AIR: a data repository for AI reliability. Specifically, we introduce key measurements and data types for assessing AI reliability, along with the methodologies used to collect these data. We also provide a detailed description of the currently available datasets with illustrative examples. Furthermore, we outline the setup of the DR-AIR repository and demonstrate its practical applications. This repository provides easy access to datasets specifically curated for AI reliability research. We believe these efforts will significantly benefit the AI research community by facilitating access to valuable reliability data and promoting collaboration across various academic domains within AI. We conclude our paper with a call to action, encouraging the research community to contribute and share AI reliability data to further advance this critical field of study.

翻译：人工智能（AI）技术与系统发展迅速。然而，确保这些系统的可靠性对于建立公众对其使用的信心至关重要。这需要对人工智能系统特有的可靠性数据进行建模与分析。人工智能可靠性研究面临的一个主要挑战，尤其是对学术界而言，是缺乏易于获取的人工智能可靠性数据。为弥补这一缺口，本文致力于对现有的人工智能可靠性数据进行全面综述，并建立DR-AIR：一个面向人工智能可靠性的数据仓库。具体而言，我们介绍了评估人工智能可靠性的关键度量指标与数据类型，以及收集这些数据所采用的方法。我们还通过示例详细描述了当前可用的数据集。此外，我们概述了DR-AIR仓库的构建方式并展示了其实际应用。该仓库为人工智能可靠性研究提供了专门整理且易于访问的数据集。我们相信，这些努力将通过促进对宝贵可靠性数据的获取以及推动人工智能内部跨学术领域的合作，使人工智能研究界显著受益。我们在文末发出行动号召，鼓励研究界贡献并分享人工智能可靠性数据，以共同推进这一关键研究领域的发展。