Artificial Intelligence (AI) and Machine Learning (ML) providers have a responsibility to develop valid and reliable systems. Much has been discussed about trusting AI and ML inferences (the process of running live data through a trained AI model to make a prediction or solve a task), but little has been done to define what that means. Those in the space of ML- based products are familiar with topics such as transparency, explainability, safety, bias, and so forth. Yet, there are no frameworks to quantify and measure those. Producing ever more trustworthy machine learning inferences is a path to increase the value of products (i.e., increased trust in the results) and to engage in conversations with users to gather feedback to improve products. In this paper, we begin by examining the dynamic of trust between a provider (Trustor) and users (Trustees). Trustors are required to be trusting and trustworthy, whereas trustees need not be trusting nor trustworthy. The challenge for trustors is to provide results that are good enough to make a trustee increase their level of trust above a minimum threshold for: 1- doing business together; 2- continuation of service. We conclude by defining and proposing a framework, and a set of viable metrics, to be used for computing a trust score and objectively understand how trustworthy a machine learning system can claim to be, plus their behavior over time.

翻译：人工智能（AI）与机器学习（ML）提供商有责任开发有效且可靠的系统。尽管关于信任AI与ML推理（即通过训练好的AI模型运行实时数据以进行预测或完成任务的过程）已有诸多讨论，但对其具体含义的界定却鲜有进展。基于ML的产品领域从业者熟悉诸如透明度、可解释性、安全性、偏见等议题，然而目前尚缺乏量化与衡量这些维度的框架。生产日益可信的机器学习推理是提升产品价值（即增强对结果的信任）的途径，也是与用户对话、收集反馈以改进产品的关键。本文首先审视了提供方（信任方）与用户（受托方）之间的信任动态：信任方需具备信任意愿且值得信赖，而受托方则不必同时满足这两点。信任方面临的挑战在于提供足够优质的结果，以使受托方将其信任水平提升至最低阈值以上，从而达成：1- 开展业务合作；2- 持续提供服务。最后，我们定义并提出了一个框架及一组可行指标，用于计算信任分数，客观理解机器学习系统可宣称的可信度及其随时间变化的行为模式。