This paper introduces FedMLSecurity, a benchmark that simulates adversarial attacks and corresponding defense mechanisms in Federated Learning (FL). As an integral module of the open-sourced library FedML that facilitates FL algorithm development and performance comparison, FedMLSecurity enhances the security assessment capacity of FedML. FedMLSecurity comprises two principal components: FedMLAttacker, which simulates attacks injected into FL training, and FedMLDefender, which emulates defensive strategies designed to mitigate the impacts of the attacks. FedMLSecurity is open-sourced 1 and is customizable to a wide range of machine learning models (e.g., Logistic Regression, ResNet, GAN, etc.) and federated optimizers (e.g., FedAVG, FedOPT, FedNOVA, etc.). Experimental evaluations in this paper also demonstrate the ease of application of FedMLSecurity to Large Language Models (LLMs), further reinforcing its versatility and practical utility in various scenarios.

翻译：本文介绍了FedMLSecurity基准测试平台，旨在模拟联邦学习（FL）中的对抗性攻击及其相应防御机制。作为开源库FedML的核心模块（该库致力于促进FL算法开发与性能对比），FedMLSecurity增强了FedML的安全评估能力。该平台包含两大组件：FedMLAttacker模拟注入FL训练过程的攻击行为，FedMLDefender则模拟旨在减轻攻击影响的防御策略。FedMLSecurity已开源发布，并可灵活适配多种机器学习模型（如逻辑回归、ResNet、GAN等）及联邦优化器（如FedAVG、FedOPT、FedNOVA等）。本文的实验评估进一步验证了FedMLSecurity在大语言模型（LLMs）中的易用性，突显其在多样化场景下的通用性与实用价值。