This work centers on the communication aspects of decentralized learning over wireless networks, using consensus-based decentralized stochastic gradient descent (D-SGD). Considering the actual communication cost or delay caused by in-network information exchange in an iterative process, our goal is to achieve fast convergence of the algorithm measured by improvement per transmission slot. We propose BASS, an efficient communication framework for D-SGD over wireless networks with broadcast transmission and probabilistic subgraph sampling. In each iteration, we activate multiple subsets of non-interfering nodes to broadcast model updates to their neighbors. These subsets are randomly activated over time, with probabilities reflecting their importance in network connectivity and subject to a communication cost constraint (e.g., the average number of transmission slots per iteration). During the consensus update step, only bi-directional links are effectively preserved to maintain communication symmetry. In comparison to existing link-based scheduling methods, the inherent broadcasting nature of wireless channels offers intrinsic advantages in speeding up convergence of decentralized learning by creating more communicated links with the same number of transmission slots.

翻译：本工作聚焦于无线网络中去中心化学习的通信方面，采用基于共识的去中心化随机梯度下降（D-SGD）算法。考虑到迭代过程中网络内信息交换带来的实际通信成本或延迟，我们的目标是实现每个传输时隙内算法改进的快速收敛。我们提出BASS，一种针对无线网络去中心化学习的高效通信框架，该框架结合广播传输与概率性子图采样。在每次迭代中，我们激活多个非干扰节点子集，使其向邻居广播模型更新。这些子集随时间随机激活，其激活概率反映了它们在网络连通性中的重要性，并受通信成本约束（例如，每迭代平均传输时隙数）。在共识更新步骤中，仅有效保留双向链路以维持通信对称性。与现有基于链路的调度方法相比，无线信道的固有广播特性通过使用相同数量的传输时隙创建更多通信链路，在加速去中心化学习收敛方面具有内在优势。