Decentralized learning (DL) is an emerging approach that enables nodes to collaboratively train a machine learning model without sharing raw data. In many application domains, such as healthcare, this approach faces challenges due to the high level of heterogeneity in the training data's feature space. Such feature heterogeneity lowers model utility and negatively impacts fairness, particularly for nodes with under-represented training data. In this paper, we introduce \textsc{Facade}, a clustering-based DL algorithm specifically designed for fair model training when the training data exhibits several distinct features. The challenge of \textsc{Facade} is to assign nodes to clusters, one for each feature, based on the similarity in the features of their local data, without requiring individual nodes to know apriori which cluster they belong to. \textsc{Facade} (1) dynamically assigns nodes to their appropriate clusters over time, and (2) enables nodes to collaboratively train a specialized model for each cluster in a fully decentralized manner. We theoretically prove the convergence of \textsc{Facade}, implement our algorithm, and compare it against three state-of-the-art baselines. Our experimental results on three datasets demonstrate the superiority of our approach in terms of model accuracy and fairness compared to all three competitors. Compared to the best-performing baseline, \textsc{Facade} on the CIFAR-10 dataset also reduces communication costs by 32.3\% to reach a target accuracy when cluster sizes are imbalanced.

翻译：去中心化学习是一种新兴方法，它使得多个节点能够在不共享原始数据的情况下协作训练机器学习模型。在许多应用领域（如医疗健康）中，由于训练数据特征空间的高度异质性，这种方法面临挑战。这种特征异质性会降低模型效用，并对公平性产生负面影响，特别是对于那些训练数据代表性不足的节点。本文提出 \textsc{Facade}，一种基于聚类的去中心化学习算法，专门针对训练数据呈现多种不同特征时的公平模型训练而设计。\textsc{Facade} 的挑战在于，根据节点本地数据特征的相似性，将节点分配到各个特征对应的聚类中，且无需单个节点预先知晓其所属的聚类。\textsc{Facade} (1) 随时间动态地将节点分配到合适的聚类，并且 (2) 使节点能够以完全去中心化的方式为每个聚类协作训练一个专用模型。我们从理论上证明了 \textsc{Facade} 的收敛性，实现了该算法，并将其与三种最先进的基线方法进行了比较。在三个数据集上的实验结果表明，在模型准确性和公平性方面，我们的方法优于所有三种对比方法。与性能最佳的基线相比，在 CIFAR-10 数据集上，当聚类规模不平衡时，\textsc{Facade} 达到目标准确率所需的通信成本也降低了 32.3\%。