In real-world machine learning (ML) pipelines, datasets are continuously growing. Models must incorporate this new training data to improve generalization and adapt to potential distribution shifts. The cost of model retraining is proportional to how frequently the model is retrained and how much data it is trained on, which makes the naive approach of retraining from scratch each time impractical. We present Modyn, a data-centric end-to-end machine learning platform. Modyn's ML pipeline abstraction enables users to declaratively describe policies for continuously training a model on a growing dataset. Modyn pipelines allow users to apply data selection policies (to reduce the number of data points) and triggering policies (to reduce the number of trainings). Modyn executes and orchestrates these continuous ML training pipelines. The system is open-source and comes with an ecosystem of benchmark datasets, models, and tooling. We formally discuss how to measure the performance of ML pipelines by introducing the concept of composite models, enabling fair comparison of pipelines with different data selection and triggering policies. We empirically analyze how various data selection and triggering policies impact model accuracy, and also show that Modyn enables high throughput training with sample-level data selection.

翻译：在实际的机器学习流水线中，数据集持续增长。模型必须整合新增的训练数据以提升泛化能力并适应潜在的分布偏移。模型重新训练的成本与模型重新训练的频次及其训练的数据量成正比，这使得每次从头开始重新训练的朴素方法变得不切实际。我们提出了Modyn，一个以数据为中心的端到端机器学习平台。Modyn的机器学习流水线抽象使用户能够以声明式方式描述在增长数据集上持续训练模型的策略。Modyn流水线允许用户应用数据选择策略（以减少数据点数量）和触发策略（以减少训练次数）。Modyn负责执行并编排这些持续的机器学习训练流水线。该系统是开源的，并附带一个包含基准数据集、模型和工具的生态系统。我们通过引入复合模型的概念，正式讨论了如何衡量机器学习流水线的性能，从而能够公平比较采用不同数据选择与触发策略的流水线。我们实证分析了各种数据选择与触发策略如何影响模型精度，并展示了Modyn能够通过样本级数据选择实现高吞吐量的训练。