Recent studies have shown that, like traditional machine learning, federated learning (FL) is also vulnerable to adversarial attacks. To improve the adversarial robustness of FL, federated adversarial training (FAT) methods have been proposed to apply adversarial training locally before global aggregation. Although these methods demonstrate promising results on independent identically distributed (IID) data, they suffer from training instability on non-IID data with label skewness, resulting in degraded natural accuracy. This tends to hinder the application of FAT in real-world applications where the label distribution across the clients is often skewed. In this paper, we study the problem of FAT under label skewness, and reveal one root cause of the training instability and natural accuracy degradation issues: skewed labels lead to non-identical class probabilities and heterogeneous local models. We then propose a Calibrated FAT (CalFAT) approach to tackle the instability issue by calibrating the logits adaptively to balance the classes. We show both theoretically and empirically that the optimization of CalFAT leads to homogeneous local models across the clients and better convergence points.

翻译：摘要：近期研究表明，联邦学习（FL）与传统机器学习一样，也易受对抗攻击影响。为提升FL的对抗鲁棒性，联邦对抗训练（FAT）方法提出在全局聚合前对本地模型进行对抗训练。尽管这些方法在独立同分布（IID）数据上展现出良好效果，但在存在标签偏斜的非独立同分布（non-IID）数据上会出现训练不稳定性，导致自然准确率下降。这阻碍了FAT在实际场景中的应用，因为实际应用中各客户端的标签分布往往存在偏斜。本文研究了标签偏斜下的FAT问题，揭示了训练不稳定与自然准确率下降的根本原因之一：偏斜标签导致非一致类别概率与非同质化本地模型。我们提出一种校准联邦对抗训练（CalFAT）方法，通过自适应校准logits以平衡类别，从而解决不稳定性问题。理论与实验均表明，CalFAT的优化能使各客户端的本地模型趋近同质化，并达到更优的收敛点。