In the Machine Learning (ML) model development lifecycle, training candidate models using an offline holdout dataset and identifying the best model for the given task is only the first step. After the deployment of the selected model, continuous model monitoring and model retraining is required in many real-world applications. There are multiple reasons for retraining, including data or concept drift, which may be reflected on the model performance as monitored by an appropriate metric. Another motivation for retraining is the acquisition of increasing amounts of data over time, which may be used to retrain and improve the model performance even in the absence of drifts. We examine the impact of various retraining decision points on crucial factors, such as model performance and resource utilization, in the context of Multilabel Classification models. We explain our key decision points and propose a reference framework for designing an effective model retraining strategy.

翻译：在机器学习模型开发生命周期中，利用离线预留数据集训练候选模型并针对特定任务遴选出最优模型仅是第一步。在选定模型部署后，众多真实世界应用场景要求持续进行模型监控与重训练。触发模型重训练的原因包括数据漂移或概念漂移等，这些变化可能通过适当的评估指标在模型性能上有所反映。另一重训练动机是随时间推移积累的数据量持续增长——即便不存在漂移现象，利用这些新增数据重新训练模型亦有助于提升性能。本研究以多标签分类模型为背景，系统考察不同重训练决策节点对模型性能与资源利用率等关键因素的影响。我们阐释了核心决策节点，并提出了用于设计高效模型重训练策略的参考框架。