Retraining machine learning models (ML) when new batches of data become available is an important task in real-world pipelines. Existing methods focus largely on greedy approaches to find the best-performing model for each batch, without considering the stability of the model's structure across retraining iterations. In this study, we propose a methodology for finding sequences of ML models that are stable across retraining iterations. We develop a mixed-integer optimization algorithm that is guaranteed to recover Pareto optimal models (in terms of the predictive power-stability trade-off) and an efficient polynomial-time algorithm that performs well in practice. Our method focuses on retaining consistent analytical insights -- which is important to model interpretability, ease of implementation, and fostering trust with users -- by using custom-defined distance metrics that can be directly incorporated into the optimization problem. Importantly, our method shows stronger stability than greedily trained models with a small, controllable sacrifice in model performance in a real-world case study. Using SHAP feature importance, we show that analytical insights are consistent across retraining iterations.

翻译：当获取新批次数据时，重训练机器学习模型（ML）是实际流水线中的关键任务。现有方法主要关注贪心策略，即寻找每批数据的最优性能模型，而未考虑模型结构在多次重训练迭代间的稳定性。本研究提出一种方法论，用于寻找在重训练迭代间保持稳定的机器学习模型序列。我们开发了混合整数优化算法，该算法能保证恢复帕累托最优模型（在预测能力与稳定性权衡方面），同时提出一种在实际应用中表现优异的高效多项式时间算法。我们的方法通过将自定义距离度量直接纳入优化问题，专注于保留一致的分析洞见——这对模型可解释性、实现便捷性及建立用户信任至关重要。重要的是，在真实案例研究中，我们的方法相比贪心训练模型展现出更强的稳定性，且模型性能仅需可控的微小牺牲。通过SHAP特征重要性分析表明，分析洞见在多次重训练迭代间保持一致性。