The zeitgeist of the digital era has been dominated by an expanding integration of Artificial Intelligence~(AI) in a plethora of applications across various domains. With this expansion, however, questions of the safety and reliability of these methods come have become more relevant than ever. Consequently, a run-time ML model safety system has been developed to ensure the model's operation within the intended context, especially in applications whose environments are greatly variable such as Autonomous Vehicles~(AVs). SafeML is a model-agnostic approach for performing such monitoring, using distance measures based on statistical testing of the training and operational datasets; comparing them to a predetermined threshold, returning a binary value whether the model should be trusted in the context of the observed data or be deemed unreliable. Although a systematic framework exists for this approach, its performance is hindered by: (1) a dependency on a number of design parameters that directly affect the selection of a safety threshold and therefore likely affect its robustness, (2) an inherent assumption of certain distributions for the training and operational sets, as well as (3) a high computational complexity for relatively large sets. This work addresses these limitations by changing the binary decision to a continuous metric. Furthermore, all data distribution assumptions are made obsolete by implementing non-parametric approaches, and the computational speed increased by introducing a new distance measure based on the Empirical Characteristics Functions~(ECF).

翻译：数字时代的精神一直由人工智能（AI）在诸多领域广泛应用的日益扩展所主导。然而，随着这一扩展，这些方法的安全性和可靠性问题变得比以往任何时候都更加重要。因此，开发了一种运行时机器学习模型安全系统，以确保模型在预期环境中的运行，特别是在环境变化剧烈的应用中，如自动驾驶车辆（AVs）。SafeML是一种与模型无关的方法，通过使用基于训练数据集和运行数据集统计检验的距离度量来执行此类监控；将这些度量与预定阈值进行比较，返回一个二元值，指示在观测数据环境下模型是否应被信任或被认为不可靠。尽管该方法已有系统化的框架，但其性能受到以下限制：（1）依赖于直接影响安全阈值选择的设计参数，从而可能影响其鲁棒性；（2）对训练集和运行集固有地假设了某种分布；（3）对于较大数据集的计算复杂度较高。本研究通过将二元决策改为连续度量来解决这些限制。此外，通过实施非参数方法，所有数据分布假设均被消除，并通过引入基于经验特征函数（ECF）的新距离度量提高了计算速度。