Federated recommender systems (FedRecs) have been widely explored recently due to their ability to protect user data privacy. In FedRecs, a central server collaboratively learns recommendation models by sharing model public parameters with clients, thereby offering a privacy-preserving solution. Unfortunately, the exposure of model parameters leaves a backdoor for adversaries to manipulate FedRecs. Existing works about FedRec security already reveal that items can easily be promoted by malicious users via model poisoning attacks, but all of them mainly focus on FedRecs with only collaborative information (i.e., user-item interactions). We argue that these attacks are effective because of the data sparsity of collaborative signals. In practice, auxiliary information, such as products' visual descriptions, is used to alleviate collaborative filtering data's sparsity. Therefore, when incorporating visual information in FedRecs, all existing model poisoning attacks' effectiveness becomes questionable. In this paper, we conduct extensive experiments to verify that incorporating visual information can beat existing state-of-the-art attacks in reasonable settings. However, since visual information is usually provided by external sources, simply including it will create new security problems. Specifically, we propose a new kind of poisoning attack for visually-aware FedRecs, namely image poisoning attacks, where adversaries can gradually modify the uploaded image to manipulate item ranks during FedRecs' training process. Furthermore, we reveal that the potential collaboration between image poisoning attacks and model poisoning attacks will make visually-aware FedRecs more vulnerable to being manipulated. To safely use visual information, we employ a diffusion model in visually-aware FedRecs to purify each uploaded image and detect the adversarial images.

翻译：联邦推荐系统因能保护用户数据隐私而近期被广泛研究。在该系统中，中心服务器通过共享模型公共参数与客户端协同学习推荐模型，从而提供隐私保护方案。然而，模型参数的暴露为攻击者操控联邦推荐系统留下了后门。现有关于联邦推荐系统安全性的研究已揭示恶意用户可通过模型投毒攻击轻松推广商品，但这些工作主要聚焦于仅包含协同信息（即用户-物品交互）的联邦推荐系统。我们认为此类攻击有效的原因在于协同信号的数据稀疏性。实践中，常利用辅助信息（如产品视觉描述）缓解协同过滤数据的稀疏性。因此，当在联邦推荐系统中引入视觉信息时，现有模型投毒攻击的有效性将存疑。本文通过大量实验验证，在合理设定下引入视觉信息可击败现有最先进的攻击方法。但由于视觉信息通常由外部来源提供，简单引入将引发新的安全问题。具体而言，我们针对视觉感知联邦推荐系统提出新型投毒攻击——图像投毒攻击，攻击者可在联邦推荐系统训练过程中逐步修改上传图片以操控物品排名。此外，我们揭示了图像投毒攻击与模型投毒攻击的潜在协同效应将使视觉感知联邦推荐系统更易被操控。为安全使用视觉信息，我们在视觉感知联邦推荐系统中引入扩散模型净化所有上传图片，并检测对抗性图像。