State-of-the-art decentralized learning algorithms typically require the data distribution to be Independent and Identically Distributed (IID). However, in practical scenarios, the data distribution across the agents can have significant heterogeneity. In this work, we propose averaging rate scheduling as a simple yet effective way to reduce the impact of heterogeneity in decentralized learning. Our experiments illustrate the superiority of the proposed method (~3% improvement in test accuracy) compared to the conventional approach of employing a constant averaging rate.

翻译：最先进的去中心化学习算法通常要求数据分布满足独立同分布（IID）条件。然而在实际场景中，各智能体间的数据分布往往存在显著异质性。本研究提出将平均率调度作为一种简单有效的方法，用于降低去中心化学习中的异质性影响。实验表明，与采用恒定平均率的传统方法相比，所提方法在测试准确率上实现了约3%的提升。