Personalization in federated learning (FL) functions as a coordinator for clients with high variance in data or behavior. Ensuring the convergence of these clients' models relies on how closely users collaborate with those with similar patterns or preferences. However, it is generally challenging to quantify similarity under limited knowledge about other users' models given to users in a decentralized network. To cope with this issue, we propose a personalized and fully decentralized FL algorithm, leveraging knowledge distillation techniques to empower each device so as to discern statistical distances between local models. Each client device can enhance its performance without sharing local data by estimating the similarity between two intermediate outputs from feeding local samples as in knowledge distillation. Our empirical studies demonstrate that the proposed algorithm improves the test accuracy of clients in fewer iterations under highly non-independent and identically distributed (non-i.i.d.) data distributions and is beneficial to agents with small datasets, even without the need for a central server.

翻译：在联邦学习（FL）中，个性化功能充当了协调器，用于处理数据或行为高度差异的客户端。确保这些客户端模型的收敛依赖于用户与具有相似模式或偏好的其他用户之间的协作紧密程度。然而，在去中心化网络中，由于用户对其他用户模型所知有限，量化相似度通常具有挑战性。为解决此问题，我们提出了一种个性化且完全去中心化的联邦学习算法，利用知识蒸馏技术使每个设备能够辨别本地模型之间的统计距离。通过类似知识蒸馏的方式，每个客户端设备无需共享本地数据，即可通过估计馈入本地样本时两个中间输出之间的相似性来提升性能。我们的实证研究表明，在高度非独立同分布（non-i.i.d.）的数据分布下，所提算法能在更少的迭代次数内提升客户端的测试准确率，并且即使无需中央服务器，也能惠及数据集较小的参与者。