Recent proliferation of powerful AI systems has created a strong need for capabilities that help users to calibrate trust in those systems. As AI systems grow in scale, information required to evaluate their trustworthiness becomes less accessible, presenting a growing risk of using these systems inappropriately. We propose the Trust Calibration Maturity Model (TCMM) to characterize and communicate information about AI system trustworthiness. The TCMM incorporates five dimensions of analytic maturity: Performance Characterization, Bias & Robustness Quantification, Transparency, Safety & Security, and Usability. The TCMM can be presented along with system performance information to (1) help a user to appropriately calibrate trust, (2) establish requirements and track progress, and (3) identify research needs. Here, we discuss the TCMM and demonstrate it on two target tasks: using ChatGPT for high consequence nuclear science determinations, and using PhaseNet (an ensemble of seismic models) for categorizing sources of seismic events.

翻译：近期强大人工智能系统的激增，催生了对帮助用户校准对这些系统信任度的能力的迫切需求。随着人工智能系统规模的扩大，评估其可信度所需的信息变得愈发难以获取，这带来了不当使用这些系统的日益增长的风险。我们提出了信任校准成熟度模型（TCMM），用于表征和传达关于人工智能系统可信度的信息。TCMM整合了分析成熟度的五个维度：性能表征、偏差与鲁棒性量化、透明度、安全性与保障性以及可用性。TCMM可与系统性能信息一同呈现，以（1）帮助用户适当校准信任，（2）建立需求并跟踪进展，以及（3）识别研究需求。本文中，我们讨论了TCMM，并在两个目标任务上进行了演示：使用ChatGPT进行高后果核科学判定，以及使用PhaseNet（一个地震模型集成）对地震事件源进行分类。