We consider the problem of decentralized optimization in networks with communication delays. To accommodate delays, we need decentralized optimization algorithms that work on directed graphs. Existing approaches require nodes to know their out-degree to achieve convergence. We propose a novel gossip-based algorithm that circumvents this requirement, allowing decentralized optimization in networks with communication delays. We prove that our algorithm converges on non-convex objectives, with the same main complexity order term as centralized Stochastic Gradient Descent (SGD), and show that the graph topology and the delays only affect the higher order terms. We provide numerical simulations that illustrate our theoretical results.

翻译：我们考虑了具有通信延迟的网络中的分布式优化问题。为应对延迟，需要能够在有向图上运行的分布式优化算法。现有方法要求节点知晓其出度才能实现收敛。我们提出了一种新颖的基于八卦的算法，规避了这一要求，使得在具有通信延迟的网络中进行分布式优化成为可能。我们证明了该算法在非凸目标上收敛，其主复杂度阶项与集中式随机梯度下降（SGD）相同，并表明图拓扑结构和延迟仅影响高阶项。我们提供了数值模拟，以验证理论结果。