In order to circumvent statistical and computational hardness results in sequential decision-making, recent work has considered smoothed online learning, where the distribution of data at each time is assumed to have bounded likeliehood ratio with respect to a base measure when conditioned on the history. While previous works have demonstrated the benefits of smoothness, they have either assumed that the base measure is known to the learner or have presented computationally inefficient algorithms applying only in special cases. This work investigates the more general setting where the base measure is \emph{unknown} to the learner, focusing in particular on the performance of Empirical Risk Minimization (ERM) with square loss when the data are well-specified and smooth. We show that in this setting, ERM is able to achieve sublinear error whenever a class is learnable with iid data; in particular, ERM achieves error scaling as $\tilde O( \sqrt{\mathrm{comp}(\mathcal F)\cdot T} )$, where $\mathrm{comp}(\mathcal F)$ is the statistical complexity of learning $\mathcal F$ with iid data. In so doing, we prove a novel norm comparison bound for smoothed data that comprises the first sharp norm comparison for dependent data applying to arbitrary, nonlinear function classes. We complement these results with a lower bound indicating that our analysis of ERM is essentially tight, establishing a separation in the performance of ERM between smoothed and iid data.

翻译：为规避序贯决策中统计与计算层面的困难结果，近期研究关注平滑在线学习问题，该设定假设每个时刻数据在给定历史条件下与基测度之间具有有界似然比。尽管已有工作证明了平滑性的优势，但这些研究要么假设学习器已知基测度，要么仅针对特殊情形提出计算低效的算法。本文研究基测度对学习器未知的更一般设定，特别关注数据满足设定规范且平滑时，平方损失函数下经验风险最小化（ERM）的表现。我们证明，在该设定下，只要类别在独立同分布数据下可学习，ERM即可实现次线性误差；具体而言，ERM的误差量级可达$\tilde O( \sqrt{\mathrm{comp}(\mathcal F)\cdot T} )$，其中$\mathrm{comp}(\mathcal F)$表示学习类别$\mathcal F$在独立同分布数据下的统计复杂度。在此过程中，我们为平滑数据证明了一个新颖的范数比较界，这是首个适用于任意非线性函数类依赖数据的尖刻范数比较结果。我们通过下界证明本文对ERM的分析本质上具有紧致性，揭示了平滑数据与独立同分布数据下ERM性能之间的差距。