Federated learning (FL) facilitates distributed training across clients, safeguarding the privacy of their data. The inherent distributed structure of FL introduces vulnerabilities, especially from adversarial (Byzantine) clients aiming to skew local updates to their advantage. Despite the plethora of research focusing on Byzantine-resilient FL, the academic community has yet to establish a comprehensive benchmark suite, pivotal for impartial assessment and comparison of different techniques. This paper investigates existing techniques in Byzantine-resilient FL and introduces an open-source benchmark suite for convenient and fair performance comparisons. Our investigation begins with a systematic study of Byzantine attack and defense strategies. Subsequently, we present \ours, a scalable, extensible, and easily configurable benchmark suite that supports researchers and developers in efficiently implementing and validating novel strategies against baseline algorithms in Byzantine-resilient FL. The design of \ours incorporates key characteristics derived from our systematic study, encompassing the attacker's capabilities and knowledge, defense strategy categories, and factors influencing robustness. Blades contains built-in implementations of representative attack and defense strategies and offers user-friendly interfaces for seamlessly integrating new ideas.

翻译：联邦学习（FL）支持跨客户端分布式训练，保障其数据隐私。FL固有的分布式结构引入了安全漏洞，尤其体现在对抗性（拜占庭）客户端试图操纵本地更新以谋取私利。尽管大量研究聚焦于拜占庭鲁棒联邦学习，学术界尚未建立全面的基准套件，这对于公平评估和比较不同技术至关重要。本文系统研究了现有拜占庭鲁棒联邦学习技术，并推出开源基准套件以促成便捷且公平的性能比较。我们首先对拜占庭攻击与防御策略展开系统研究，进而提出\ours——一个可扩展、可延伸且易配置的基准套件，支持研究人员与开发者高效实现并验证针对拜占庭鲁棒FL基线算法的新策略。\ours的设计融合了系统研究中的关键特征，包括攻击者能力与知识、防御策略分类及影响鲁棒性的因素。刀锋内置了代表性攻击与防御策略的即用实现，并提供用户友好接口以无缝集成新思路。