The widespread use of machine learning algorithms calls for automatic change detection algorithms to monitor their behavior over time. As a machine learning algorithm learns from a continuous, possibly evolving, stream of data, it is desirable and often critical to supplement it with a companion change detection algorithm to facilitate its monitoring and control. We present a generic score-based change detection method that can detect a change in any number of components of a machine learning model trained via empirical risk minimization. This proposed statistical hypothesis test can be readily implemented for such models designed within a differentiable programming framework. We establish the consistency of the hypothesis test and show how to calibrate it to achieve a prescribed false alarm rate. We illustrate the versatility of the approach on synthetic and real data.

翻译：广泛使用机器学习算法需要自动变化检测算法来监测其长期行为。随着机器学习算法从连续的、可能演进的、不断演变的数据流中学习,最好而且往往是关键的是补充这一算法,以辅助其变化检测算法,便利其监测和控制。我们提出了一个通用的基于分数的变化检测方法,该方法可以检测通过尽量减少风险的经验培训的机器学习模型中任何部分的变化。这个提议的统计假设测试可以很容易地用于在不同的编程框架内设计的模型。我们建立了假设测试的一致性,并展示了如何校准它以达到规定的虚假警报率。我们展示了合成数据和真实数据方法的多功能性。