In real-world applications, the distribution of the data, and our goals, evolve over time. The prevailing theoretical framework for studying machine learning, namely probably approximately correct (PAC) learning, largely ignores time. As a consequence, existing strategies to address the dynamic nature of data and goals exhibit poor real-world performance. This paper develops a theoretical framework called "Prospective Learning" that is tailored for situations when the optimal hypothesis changes over time. In PAC learning, empirical risk minimization (ERM) is known to be consistent. We develop a learner called Prospective ERM, which returns a sequence of predictors that make predictions on future data. We prove that the risk of prospective ERM converges to the Bayes risk under certain assumptions on the stochastic process generating the data. Prospective ERM, roughly speaking, incorporates time as an input in addition to the data. We show that standard ERM as done in PAC learning, without incorporating time, can result in failure to learn when distributions are dynamic. Numerical experiments illustrate that prospective ERM can learn synthetic and visual recognition problems constructed from MNIST and CIFAR-10. Code at https://github.com/neurodata/prolearn.

翻译：在实际应用中，数据的分布以及我们的目标会随时间演变。当前研究机器学习的主流理论框架——可能近似正确（PAC）学习——在很大程度上忽略了时间因素。因此，现有应对数据和目标动态特性的策略在现实世界中表现不佳。本文提出了一种名为“前瞻性学习”的理论框架，专门针对最优假设随时间变化的情形。在PAC学习中，经验风险最小化（ERM）已知具有一致性。我们提出了一种称为前瞻性ERM的学习器，它返回一系列对未来数据进行预测的预测器。我们证明，在数据生成随机过程的某些假设下，前瞻性ERM的风险会收敛到贝叶斯风险。粗略地说，前瞻性ERM除了数据之外还将时间作为输入。我们表明，PAC学习中实施的标准ERM若不纳入时间因素，在分布动态变化时可能导致学习失败。数值实验表明，前瞻性ERM能够学习基于MNIST和CIFAR-10构建的合成及视觉识别问题。代码位于 https://github.com/neurodata/prolearn。