Covariate shift may impact the operational safety performance of neural networks. A re-evaluation of the safety performance, however, requires collecting new operational data and creating corresponding ground truth labels, which often is not possible during operation. We are therefore proposing to reshape the initial test set, as used for the safety performance evaluation prior to deployment, based on an approximation of the operational data. This approximation is obtained by observing and learning the distribution of activation patterns of neurons in the network during operation. The reshaped test set reflects the distribution of neuron activation values as observed during operation, and may therefore be used for re-evaluating safety performance in the presence of covariate shift. First, we derive conservative bounds on the values of neurons by applying finite binning and static dataflow analysis. Second, we formulate a mixed integer linear programming (MILP) constraint for constructing the minimum set of data points to be removed in the test set, such that the difference between the discretized test and operational distributions is bounded. We discuss potential benefits and limitations of this constraint-based approach based on our initial experience with an implemented research prototype.

翻译：协变量偏移可能影响神经网络的操作安全性表现。然而，重新评估其安全性表现需要收集新的操作数据并创建相应的真实标签，这在操作过程中通常难以实现。因此，我们提出基于操作数据的近似估计，重塑部署前用于安全性评估的初始测试集。该近似估计通过观测并学习网络操作过程中神经元激活模式的分布获得。重塑后的测试集反映了操作期间观测到的神经元激活值分布，从而可用于在协变量偏移下重新评估安全性表现。首先，我们通过有限分箱和静态数据流分析，推导神经元值的保守界。其次，我们构建混合整数线性规划（MILP）约束，用于确定测试集中需移除的最小数据点集合，以使离散化测试分布与操作分布之间的差异受限于给定边界。基于已实现的研究原型初步经验，我们讨论了这种基于约束的方法的潜在优势与局限性。