This paper introduces FedMLSecurity, a benchmark designed to simulate adversarial attacks and corresponding defense mechanisms in Federated Learning (FL). As an integral module of the open-sourced library FedML that facilitates FL algorithm development and performance comparison, FedMLSecurity enhances FedML's capabilities to evaluate security issues and potential remedies in FL. FedMLSecurity comprises two major components: FedMLAttacker that simulates attacks injected during FL training, and FedMLDefender that simulates defensive mechanisms to mitigate the impacts of the attacks. FedMLSecurity is open-sourced and can be customized to a wide range of machine learning models (e.g., Logistic Regression, ResNet, GAN, etc.) and federated optimizers (e.g., FedAVG, FedOPT, FedNOVA, etc.). FedMLSecurity can also be applied to Large Language Models (LLMs) easily, demonstrating its adaptability and applicability in various scenarios.

翻译：本文介绍了FedMLSecurity基准测试，该测试专为模拟联邦学习（FL）中的对抗性攻击及相应防御机制而设计。作为开源库FedML的核心模块（该库旨在促进FL算法开发与性能比较），FedMLSecurity增强了FedML在评估FL安全问题及潜在解决方案方面的能力。FedMLSecurity包含两大组件：FedMLAttacker模拟FL训练过程中注入的攻击，FedMLDefender则模拟防御机制以减轻攻击影响。FedMLSecurity为开源框架，可灵活适配各类机器学习模型（如逻辑回归、ResNet、生成对抗网络等）及联邦优化器（如FedAVG、FedOPT、FedNOVA等）。此外，该基准测试还可便捷应用于大语言模型（LLMs），充分展现其在不同场景下的适应性与实用性。