In the Machine Learning (ML) model development lifecycle, training candidate models using an offline holdout dataset and identifying the best model for the given task is only the first step. After the deployment of the selected model, continuous model monitoring and model retraining is required in many real-world applications. There are multiple reasons for retraining, including data or concept drift, which may be reflected on the model performance as monitored by an appropriate metric. Another motivation for retraining is the acquisition of increasing amounts of data over time, which may be used to retrain and improve the model performance even in the absence of drifts. We examine the impact of various retraining decision points on crucial factors, such as model performance and resource utilization, in the context of Multilabel Classification models. We explain our key decision points and propose a reference framework for designing an effective model retraining strategy.

翻译：在机器学习模型开发生命周期中，使用离线留存数据集训练候选模型并针对特定任务筛选最佳模型仅是第一步。模型部署后，众多实际应用场景需要持续进行模型监控与重训练。重训练的诱因包括数据漂移或概念漂移——这些现象可能通过特定指标监测到的模型性能变化得以体现。另一重训练驱动因素在于随时间推移积累的增量数据，即便未发生漂移现象，这些数据也可用于重训练以提升模型性能。本文以多标签分类模型为背景，系统研究了不同重训练决策时间点对模型性能与资源利用率等关键要素的影响，阐释了核心决策节点的选择依据，并提出了一个用于设计高效模型重训练策略的参考框架。