Decentralized learning (DL) has gained prominence for its potential benefits in terms of scalability, privacy, and fault tolerance. It consists of many nodes that coordinate without a central server and exchange millions of parameters in the inherently iterative process of machine learning (ML) training. In addition, these nodes are connected in complex and potentially dynamic topologies. Assessing the intricate dynamics of such networks is clearly not an easy task. Often in literature, researchers resort to simulated environments that do not scale and fail to capture practical and crucial behaviors, including the ones associated to parallelism, data transfer, network delays, and wall-clock time. In this paper, we propose DecentralizePy, a distributed framework for decentralized ML, which allows for the emulation of large-scale learning networks in arbitrary topologies. We demonstrate the capabilities of DecentralizePy by deploying techniques such as sparsification and secure aggregation on top of several topologies, including dynamic networks with more than one thousand nodes.

翻译：去中心化学习（Decentralized Learning，DL）因其在可扩展性、隐私性和容错性方面的潜在优势而受到广泛关注。该过程由众多节点组成，节点之间无需中央服务器即可协调，并在机器学习（ML）训练这一固有迭代过程中交换数百万个参数。此外，这些节点以复杂且可能动态变化的拓扑结构相互连接。评估此类网络的复杂动力学特性显然并非易事。文献中，研究人员常借助模拟环境，但这些环境不仅扩展性差，且无法捕捉实际中关键的行为特征，例如与并行性、数据传输、网络延迟及挂钟时间相关的行为。本文提出DecentralizePy——一个用于去中心化机器学习的分布式框架，该框架支持在任意拓扑结构下模拟大规模学习网络。我们通过在包含超过一千个节点的动态网络等多种拓扑结构上部署稀疏化（sparsification）和安全聚合（secure aggregation）等技术，展示了DecentralizePy的能力。