Membership Inference Attacks (MIAs) pose a growing threat to privacy preservation in federated learning. The semi-honest attacker, e.g., the server, may determine whether a particular sample belongs to a target client according to the observed model information. This paper conducts an evaluation of existing MIAs and corresponding defense strategies. Our evaluation on MIAs reveals two important findings about the trend of MIAs. Firstly, combining model information from multiple communication rounds (Multi-temporal) enhances the overall effectiveness of MIAs compared to utilizing model information from a single epoch. Secondly, incorporating models from non-target clients (Multi-spatial) significantly improves the effectiveness of MIAs, particularly when the clients' data is homogeneous. This highlights the importance of considering the temporal and spatial model information in MIAs. Next, we assess the effectiveness via privacy-utility tradeoff for two type defense mechanisms against MIAs: Gradient Perturbation and Data Replacement. Our results demonstrate that Data Replacement mechanisms achieve a more optimal balance between preserving privacy and maintaining model utility. Therefore, we recommend the adoption of Data Replacement methods as a defense strategy against MIAs. Our code is available in https://github.com/Liar-Mask/FedMIA.

翻译：成员推断攻击对联邦学习中的隐私保护构成日益严重的威胁。半诚实的攻击者（如服务器）可能根据观察到的模型信息，判断特定样本是否属于目标客户端。本文对现有成员推断攻击及其对应防御策略进行了评估。我们对成员推断攻击的评估揭示了两项重要发现：首先，相较于利用单一训练轮次的模型信息，结合多个通信轮次的模型信息（跨时间维度）能够提升成员推断攻击的整体有效性；其次，整合非目标客户端的模型信息（跨空间维度）可显著增强成员推断攻击效果，尤其是在客户端数据同质的情况下。这凸显了在成员推断攻击中考虑时空模型信息的重要性。随后，我们通过隐私-效用权衡评估了两类针对成员推断攻击的防御机制：梯度扰动与数据替换。研究结果表明，数据替换机制在保护隐私与维持模型效用之间取得了更优平衡。因此，我们建议采用数据替换方法作为抵御成员推断攻击的防御策略。我们的代码已在https://github.com/Liar-Mask/FedMIA 开源。