It is important to retrain a machine learning (ML) model in order to maintain its performance as the data changes over time. However, this can be costly as it usually requires processing the entire dataset again. This creates a trade-off between retraining too frequently, which leads to unnecessary computing costs, and not retraining often enough, which results in stale and inaccurate ML models. To address this challenge, we propose ML systems that make automated and cost-effective decisions about when to retrain an ML model. We aim to optimize the trade-off by considering the costs associated with each decision. Our research focuses on determining whether to retrain or keep an existing ML model based on various factors, including the data, the model, and the predictive queries answered by the model. Our main contribution is a Cost-Aware Retraining Algorithm called Cara, which optimizes the trade-off over streams of data and queries. To evaluate the performance of Cara, we analyzed synthetic datasets and demonstrated that Cara can adapt to different data drifts and retraining costs while performing similarly to an optimal retrospective algorithm. We also conducted experiments with real-world datasets and showed that Cara achieves better accuracy than drift detection baselines while making fewer retraining decisions, ultimately resulting in lower total costs.

翻译：为了在数据随时间变化时保持机器学习（ML）模型的性能，对其进行重训练至关重要。然而，这通常需要重新处理整个数据集，从而带来高昂的成本。这便在频繁重训练（导致不必要的计算成本）与不足量重训练（导致模型过时且不准确）之间形成了权衡。为应对这一挑战，本文提出了一套ML系统，能够自动化地、以成本效益为目标地决策何时对ML模型进行重训练。我们旨在通过考量每次决策所关联的成本来优化这一权衡。研究聚焦于根据数据、模型及模型所响应的预测查询等多重因素，决定是重训练还是保留现有ML模型。主要贡献在于提出了一种名为Cara的成本感知重训练算法，该算法能针对数据流与查询流优化上述权衡。为评估Cara的性能，我们通过合成数据集进行分析，证明Cara能适应不同的数据漂移与重训练成本，其表现与最优的回顾性算法相当。此外，基于真实数据集的实验表明，Cara在减少重训练决策次数的同时，相较于漂移检测基线方法取得了更优的准确率，最终实现了更低的总成本。