Machine learning models are famously vulnerable to adversarial attacks: small ad-hoc perturbations of the data that can catastrophically alter the model predictions. While a large literature has studied the case of test-time attacks on pre-trained models, the important case of attacks in an online learning setting has received little attention so far. In this work, we use a control-theoretical perspective to study the scenario where an attacker may perturb data labels to manipulate the learning dynamics of an online learner. We perform a theoretical analysis of the problem in a teacher-student setup, considering different attack strategies, and obtaining analytical results for the steady state of simple linear learners. These results enable us to prove that a discontinuous transition in the learner's accuracy occurs when the attack strength exceeds a critical threshold. We then study empirically attacks on learners with complex architectures using real data, confirming the insights of our theoretical analysis. Our findings show that greedy attacks can be extremely efficient, especially when data stream in small batches.

翻译：机器学习模型因其对抗性攻击而闻名：对数据的小规模特定扰动能够灾难性地改变模型预测。尽管大量文献研究了预训练模型在测试时的攻击情况，但在线学习环境下的攻击这一重要情况迄今很少受到关注。本文中，我们采用控制理论的视角来研究攻击者可能通过扰动数据标签来操纵在线学习者的学习动态的场景。我们在师生框架下对该问题进行理论分析，考虑不同的攻击策略，并针对简单线性学习者的稳态获得了解析结果。这些结果使我们能够证明，当攻击强度超过临界阈值时，学习者的准确性会发生不连续转变。随后，我们使用真实数据对具有复杂结构的学习者的攻击进行了实证研究，证实了我们理论分析的见解。我们的研究结果表明，贪婪攻击可以极为高效，特别是在数据以小批量流式传输时。