This paper explores decentralized learning in a graph-based setting, where data is distributed across nodes. We investigate a decentralized SGD algorithm that utilizes a random walk to update a global model based on local data. Our focus is on designing the transition probability matrix to speed up convergence. While importance sampling can enhance centralized learning, its decentralized counterpart, using the Metropolis-Hastings (MH) algorithm, can lead to the entrapment problem, where the random walk becomes stuck at certain nodes, slowing convergence. To address this, we propose the Metropolis-Hastings with L\'evy Jumps (MHLJ) algorithm, which incorporates random perturbations (jumps) to overcome entrapment. We theoretically establish the convergence rate and error gap of MHLJ and validate our findings through numerical experiments.

翻译：本文研究基于图结构的去中心化学习，其中数据分布在各个节点上。我们探讨了一种利用随机游走根据本地数据更新全局模型的去中心化随机梯度下降算法。研究重点在于设计转移概率矩阵以加速收敛。虽然重要性采样能够提升中心化学习的性能，但其去中心化版本——使用Metropolis-Hastings算法的方案——可能导致陷阱问题，即随机游走被困在某些节点，从而延缓收敛速度。为解决此问题，我们提出了带莱维跳跃的Metropolis-Hastings算法，该算法通过引入随机扰动来克服陷阱效应。我们从理论上证明了该算法的收敛速度与误差界，并通过数值实验验证了研究结论。