Personalization and decentralization are two major lines of studies to realize practical federated learning in the real world. The aim of this study is to establish a general and unified approach that can solve these two problems simultaneously. In this work, we first propose a bilevel problem that can adapt to various personalization scenarios by allowing an arbitrary choice of two parameters: a client-wise outer-parameter representing heterogeneity, and a shared inner-parameter representing homogeneity across client data distributions. We then present an algorithm that can solve this bilevel problem in a decentralized manner by estimating gradients of clients' outer-costs with respect to their outer-parameters. We show that the proposed algorithm can be extended to handle a random directed network, which is one of the most robust decentralized communication classes. The proposed method achieves state-of-the-art performance on a personalization benchmark across various communication settings.

翻译：个性化和去中心化是实现现实世界中联邦学习的两条主要研究方向。本研究旨在建立一种通用且统一的方法，以同时解决这两个问题。本文首先提出一个双层优化问题，通过允许任意选择两个参数（表示客户端数据分布异质性的客户端级外部参数和表示同质性的共享内部参数），能够适应多种个性化场景。随后，我们提出一种算法，通过估计客户端外部成本相对于其外部参数的梯度，以去中心化方式求解该双层问题。研究表明，所提算法可扩展至处理随机有向网络——这是最鲁棒的去中心化通信类别之一。所提方法在多种通信设置下的个性化基准测试中实现了最先进的性能。