We study the generalization error of statistical learning algorithms in a non-i.i.d. setting, where the training data is sampled from a stationary mixing process. We develop an analytic framework for this scenario based on a reduction to online learning with delayed feedback. In particular, we show that the existence of an online learning algorithm with bounded regret (against a fixed statistical learning algorithm in a specially constructed game of online learning with delayed feedback) implies low generalization error of said statistical learning method even if the data sequence is sampled from a mixing time series. The rates demonstrate a trade-off between the amount of delay in the online learning game and the degree of dependence between consecutive data points, with near-optimal rates recovered in a number of well-studied settings when the delay is tuned appropriately as a function of the mixing time of the process.

翻译：我们研究了统计学习算法在非独立同分布设置下的泛化误差，其中训练数据来自平稳混合过程。我们基于延迟反馈在线学习的归约方法，为此场景开发了一个分析框架。具体而言，我们证明了：若存在一种在线学习算法（在特定构造的延迟反馈在线学习博弈中，相对于固定统计学习方法）具有有界遗憾，则即使数据序列来自混合时间序列，该统计学习方法仍具有较低的泛化误差。所得速率揭示了在线学习博弈中的延迟量与连续数据点间依赖程度之间的权衡关系：当延迟量根据过程的混合时间适当调整时，可在多个经典研究场景中恢复接近最优的速率。