Owing to the nature of privacy protection, federated recommender systems (FedRecs) have garnered increasing interest in the realm of on-device recommender systems. However, most existing FedRecs only allow participating clients to collaboratively train a recommendation model of the same public parameter size. Training a model of the same size for all clients can lead to suboptimal performance since clients possess varying resources. For example, clients with limited training data may prefer to train a smaller recommendation model to avoid excessive data consumption, while clients with sufficient data would benefit from a larger model to achieve higher recommendation accuracy. To address the above challenge, this paper introduces HeteFedRec, a novel FedRec framework that enables the assignment of personalized model sizes to participants. In HeteFedRec, we present a heterogeneous recommendation model aggregation strategy, including a unified dual-task learning mechanism and a dimensional decorrelation regularization, to allow knowledge aggregation among recommender models of different sizes. Additionally, a relation-based ensemble knowledge distillation method is proposed to effectively distil knowledge from heterogeneous item embeddings. Extensive experiments conducted on three real-world recommendation datasets demonstrate the effectiveness and efficiency of HeteFedRec in training federated recommender systems under heterogeneous settings.

翻译：由于隐私保护的特性，联邦推荐系统在设备端推荐领域日益受到关注。然而，现有大多数联邦推荐系统仅允许参与客户端协同训练一个相同公共参数规模的推荐模型。为所有客户端训练同等规模的模型可能导致性能欠优——例如，训练数据有限的客户端更倾向于训练较小规模的推荐模型以避免过度消耗数据，而数据充足的客户端则需借助更大规模模型实现更高推荐精度。针对上述挑战，本文提出HeteFedRec——一种新型联邦推荐框架，能够为参与者分配个性化模型规模。在HeteFedRec中，我们提出一种异质性推荐模型聚合策略，包含统一双任务学习机制与维度去相关正则化，以实现不同规模推荐模型间的知识聚合。此外，提出基于关系的集成知识蒸馏方法，有效蒸馏异质性项目嵌入中的知识。在三个真实推荐数据集上的大量实验表明，HeteFedRec在异质性场景下训练联邦推荐系统时兼具高效性与有效性。