Real-world deployment of machine learning models is challenging when data evolves over time. And data does evolve over time. While no model can work when data evolves in an arbitrary fashion, if there is some pattern to these changes, we might be able to design methods to address it. This paper addresses situations when data evolves gradually. We introduce a novel time-varying importance weight estimator that can detect gradual shifts in the distribution of data. Such an importance weight estimator allows the training method to selectively sample past data -- not just similar data from the past like a standard importance weight estimator would but also data that evolved in a similar fashion in the past. Our time-varying importance weight is quite general. We demonstrate different ways of implementing it that exploit some known structure in the evolution of data. We demonstrate and evaluate this approach on a variety of problems ranging from supervised learning tasks (multiple image classification datasets) where the data undergoes a sequence of gradual shifts of our design to reinforcement learning tasks (robotic manipulation and continuous control) where data undergoes a shift organically as the policy or the task changes.

翻译：机器学习模型在实际部署中面临数据随时间演化的挑战。虽然任意演化的数据会使任何模型失效，但若这些变化存在某种模式，我们或许能够设计相应方法加以应对。本文针对数据渐进式演化的情况展开研究。我们提出了一种新颖的时变重要性权重估计器，能够检测数据分布的渐进式漂移。这类重要性权重估计器允许训练方法有选择性地对历史数据进行采样——不仅是如传统重要性权重估计器那样选取与当前分布相似的历史数据，更能提取过去以类似方式演化的数据。我们的时变重要性权重具有高度通用性。我们展示了利用数据演化中已知结构特征实现该权重的多种方式，并在监督学习任务（涵盖经过设计序列化渐进漂移的多个图像分类数据集）与强化学习任务（包含因策略或任务变化产生有机数据漂移的机器人操作与连续控制场景）中对该方法进行了验证与评估。