We study decentralized learning over networks where data are distributed across nodes without a central coordinator. Random walk learning is a token-based approach in which a single model is propagated across the network and updated at each visited node using local data, thereby incurring low communication and computational overheads. In weighted random-walk learning, the transition matrix is designed to achieve a desired sampling distribution, thereby speeding up convergence under data heterogeneity. We show that implementing weighted sampling via the Metropolis-Hastings algorithm can lead to a previously unexplored phenomenon we term entrapment. The random walk may become trapped in a small region of the network, resulting in highly correlated updates and severely degraded convergence. To address this issue, we propose Metropolis-Hastings with Levy jumps, which introduces occasional long-range transitions to restore exploration while respecting local information constraints. We establish a convergence rate that explicitly characterizes the roles of data heterogeneity, network spectral gap, and jump probability, and demonstrate through experiments that MHLJ effectively eliminates entrapment and significantly speeds up decentralized learning.

翻译：我们研究去中心化网络上的学习问题，其中数据分布在各个节点上，且没有中央协调器。随机游走学习是一种基于令牌的方法，其中单个模型在网络中传播，并在每个访问节点上利用本地数据进行更新，从而带来较低的通信和计算开销。在加权随机游走学习中，转移矩阵被设计为实现所需的采样分布，从而在数据异质性下加快收敛速度。我们表明，通过Metropolis-Hastings算法实现加权采样可能导致一种此前未被探索的现象，我们称之为“困陷”。随机游走可能被困在网络的一个小区域内，导致高度相关的更新，严重降低收敛速度。为解决这一问题，我们提出了带有莱维跳跃的Metropolis-Hastings方法，该方法引入偶尔的长距离跳转，以在遵守本地信息约束的同时恢复探索。我们建立了显式表征数据异质性、网络谱隙和跳跃概率作用的收敛速率，并通过实验证明，MHLJ有效消除了困陷，显著加快了去中心化学习的速度。