Decentralized learning has recently been attracting increasing attention for its applications in parallel computation and privacy preservation. Many recent studies stated that the underlying network topology with a faster consensus rate (a.k.a. spectral gap) leads to a better convergence rate and accuracy for decentralized learning. However, a topology with a fast consensus rate, e.g., the exponential graph, generally has a large maximum degree, which incurs significant communication costs. Thus, seeking topologies with both a fast consensus rate and small maximum degree is important. In this study, we propose a novel topology combining both a fast consensus rate and small maximum degree called the Base-$(k + 1)$ Graph. Unlike the existing topologies, the Base-$(k + 1)$ Graph enables all nodes to reach the exact consensus after a finite number of iterations for any number of nodes and maximum degree k. Thanks to this favorable property, the Base-$(k + 1)$ Graph endows Decentralized SGD (DSGD) with both a faster convergence rate and more communication efficiency than the exponential graph. We conducted experiments with various topologies, demonstrating that the Base-$(k + 1)$ Graph enables various decentralized learning methods to achieve higher accuracy with better communication efficiency than the existing topologies.

翻译：去中心化学习因其在并行计算与隐私保护中的应用而日益受到关注。近期研究表明，具有更快共识速率（即谱隙）的底层网络拓扑能够提升去中心化学习的收敛速率与精度。然而，具备快速共识速率的拓扑结构（如指数图）通常具有较大的最大度数，从而导致显著的通信开销。因此，寻找兼具快速共识速率与小最大度数的拓扑至关重要。本研究提出一种新型拓扑——Base-$(k + 1)$ 图，其同时具备快速共识速率与小最大度数。与现有拓扑不同，Base-$(k + 1)$ 图可使任意节点数及最大度数 $k$ 下的所有节点在有限迭代次数后达到精确共识。凭借这一优良特性，与指数图相比，Base-$(k + 1)$ 图赋予了去中心化随机梯度下降（DSGD）更快的收敛速率与更高的通信效率。我们基于多种拓扑进行了实验，结果表明，相较于现有拓扑，Base-$(k + 1)$ 图能使各类去中心化学习方法在提升通信效率的同时实现更高精度。