Federated recommendations (FRs), facilitating multiple local clients to collectively learn a global model without disclosing user private data, have emerged as a prevalent architecture for privacy-preserving recommendations. In conventional FRs, a dominant paradigm is to utilize discrete identities to represent users/clients and items, which are subsequently mapped to domain-specific embeddings to participate in model training. Despite considerable performance, we reveal three inherent limitations that can not be ignored in federated settings, i.e., non-transferability across domains, unavailability in cold-start settings, and potential privacy violations during federated training. To this end, we propose a transferable federated recommendation model with universal textual representations, TransFR, which delicately incorporates the general capabilities empowered by pre-trained language models and the personalized abilities by fine-tuning local private data. Specifically, it first learns domain-agnostic representations of items by exploiting pre-trained models with public textual corpora. To tailor for federated recommendation, we further introduce an efficient federated fine-tuning and a local training mechanism. This facilitates personalized local heads for each client by utilizing their private behavior data. By incorporating pre-training and fine-tuning within FRs, it greatly improves the adaptation efficiency transferring to a new domain and the generalization capacity to address cold-start issues. Through extensive experiments on several datasets, we demonstrate that our TransFR model surpasses several state-of-the-art FRs in terms of accuracy, transferability, and privacy.

翻译：联邦推荐（FRs）允许多个本地客户端协同学习全局模型而不泄露用户隐私数据，已成为隐私保护推荐领域的流行架构。传统FRs的主要范式是利用离散身份标识表示用户/客户端和物品，并将其映射为领域特定的嵌入向量参与模型训练。尽管性能显著，我们发现联邦场景中存在三个固有局限性：跨领域不可迁移性、冷启动场景不可用性及联邦训练中潜在的隐私泄露风险。为此，我们提出基于通用文本表示的可迁移联邦推荐模型TransFR，该模型巧妙融合预训练语言模型赋予的通用能力与本地私有数据微调获得的个性化能力。具体而言，它首先利用公开文本语料库的预训练模型学习物品的领域无关表示。为适配联邦推荐场景，我们进一步引入高效联邦微调和本地训练机制，使各客户端能利用私有行为数据实现个性化本地头部网络。通过将预训练与微调融入FRs，该方法显著提升了迁移至新领域的适应效率与解决冷启动问题的泛化能力。多数据集上的大量实验表明，我们的TransFR模型在准确性、可迁移性和隐私保护方面均优于多个现有最优联邦推荐模型。