Owing to the nature of privacy protection, federated recommender systems (FedRecs) have garnered increasing interest in the realm of on-device recommender systems. However, most existing FedRecs only allow participating clients to collaboratively train a recommendation model of the same public parameter size. Training a model of the same size for all clients can lead to suboptimal performance since clients possess varying resources. For example, clients with limited training data may prefer to train a smaller recommendation model to avoid excessive data consumption, while clients with sufficient data would benefit from a larger model to achieve higher recommendation accuracy. To address the above challenge, this paper introduces HeteFedRec, a novel FedRec framework that enables the assignment of personalized model sizes to participants. In HeteFedRec, we present a heterogeneous recommendation model aggregation strategy, including a unified dual-task learning mechanism and a dimensional decorrelation regularization, to allow knowledge aggregation among recommender models of different sizes. Additionally, a relation-based ensemble knowledge distillation method is proposed to effectively distil knowledge from heterogeneous item embeddings. Extensive experiments conducted on three real-world recommendation datasets demonstrate the effectiveness and efficiency of HeteFedRec in training federated recommender systems under heterogeneous settings.

翻译：由于隐私保护的特性，联邦推荐系统（FedRecs）在设备端推荐系统领域日益受到关注。然而，现有大多数FedRecs仅允许参与客户端协作训练具有相同公共参数规模的推荐模型。为所有客户端训练相同规模的模型可能导致性能次优，因为不同客户端的资源存在差异。例如，训练数据有限的客户端可能更倾向于训练较小的推荐模型以避免过度消耗数据，而数据充足的客户端则能从更大模型中获益以实现更高的推荐精度。为解决上述挑战，本文提出HeteFedRec——一种新颖的FedRec框架，能够为参与者分配个性化的模型规模。在HeteFedRec中，我们提出了一种异构推荐模型聚合策略，包括统一双任务学习机制和维度去相关正则化，以实现不同规模推荐模型间的知识聚合。此外，还提出了一种基于关系的集成知识蒸馏方法，有效蒸馏异构项目嵌入中的知识。在三个真实推荐数据集上的大量实验表明，HeteFedRec在异构设置下训练联邦推荐系统具有高效性和有效性。