We study the extent to which standard machine learning algorithms rely on exchangeability and independence of data by introducing a monotone adversarial corruption model. In this model, an adversary, upon looking at a "clean" i.i.d. dataset, inserts additional "corrupted" points of their choice into the dataset. These added points are constrained to be monotone corruptions, in that they get labeled according to the ground-truth target function. Perhaps surprisingly, we demonstrate that in this setting, all known optimal learning algorithms for binary classification can be made to achieve suboptimal expected error on a new independent test point drawn from the same distribution as the clean dataset. On the other hand, we show that uniform convergence-based algorithms do not degrade in their guarantees. Our results showcase how optimal learning algorithms break down in the face of seemingly helpful monotone corruptions, exposing their overreliance on exchangeability.

翻译：本文通过引入单调对抗性污染模型，研究标准机器学习算法在多大程度上依赖于数据的可交换性和独立性。在该模型中，敌手在观察到一个"干净"的独立同分布数据集后，可以自主选择插入额外的"污染"数据点。这些添加的点被约束为单调污染，即它们根据真实目标函数进行标注。令人惊讶的是，我们证明在此设定下，所有已知的二元分类最优学习算法对于从与干净数据集相同分布中抽取的新独立测试点，都可能获得次优的期望误差。另一方面，我们证明基于一致收敛的算法不会降低其性能保证。我们的结果揭示了最优学习算法在面对看似有益的单调污染时如何失效，暴露了它们对可交换性的过度依赖。