Prediction models frequently face the challenge of concept drift, in which the underlying data distribution changes over time, weakening performance. Examples can include models which predict loan default, or those used in healthcare contexts. Typical management strategies involve regular model updates or updates triggered by concept drift detection. However, these simple policies do not necessarily balance the cost of model updating with improved classifier performance. We present AMUSE (Adaptive Model Updating using a Simulated Environment), a novel method leveraging reinforcement learning trained within a simulated data generating environment, to determine update timings for classifiers. The optimal updating policy depends on the current data generating process and ongoing drift process. Our key idea is that we can train an arbitrarily complex model updating policy by creating a training environment in which possible episodes of drift are simulated by a parametric model, which represents expectations of possible drift patterns. As a result, AMUSE proactively recommends updates based on estimated performance improvements, learning a policy that balances maintaining model performance with minimizing update costs. Empirical results confirm the effectiveness of AMUSE in simulated data.

翻译：预测模型常面临概念漂移的挑战，即底层数据分布随时间变化导致性能下降。此类问题常见于贷款违约预测模型或医疗健康领域的预测模型。典型的管理策略包括定期模型更新或基于概念漂移检测的触发式更新。然而，这些简单策略未必能平衡模型更新成本与分类器性能提升之间的关系。本文提出AMUSE（基于模拟环境的自适应模型更新方法），这是一种利用在模拟数据生成环境中训练的强化学习来确定分类器更新时机的新方法。最优更新策略取决于当前数据生成过程与持续漂移过程。我们的核心思想是：通过创建训练环境来训练任意复杂的模型更新策略，在该环境中，参数化模型模拟可能发生的漂移事件，该参数化模型表征了对潜在漂移模式的预期。因此，AMUSE能基于预估的性能改进主动推荐更新，学习到平衡模型性能维持与更新成本最小化的策略。实验结果表明AMUSE在模拟数据中的有效性。