Fully decentralised federated learning enables collaborative training of individual machine learning models on distributed devices on a network while keeping the training data localised. This approach enhances data privacy and eliminates both the single point of failure and the necessity for central coordination. Our research highlights that the effectiveness of decentralised federated learning is significantly influenced by the network topology of connected devices. A simplified numerical model for studying the early behaviour of these systems leads us to an improved artificial neural network initialisation strategy, which leverages the distribution of eigenvector centralities of the nodes of the underlying network, leading to a radically improved training efficiency. Additionally, our study explores the scaling behaviour and choice of environmental parameters under our proposed initialisation strategy. This work paves the way for more efficient and scalable artificial neural network training in a distributed and uncoordinated environment, offering a deeper understanding of the intertwining roles of network structure and learning dynamics.
翻译:完全去中心化的联邦学习能够在网络上的分布式设备上协作训练个体机器学习模型,同时保持训练数据的本地化。这种方法增强了数据隐私保护,消除了单点故障和集中协调的必要性。我们的研究强调,去中心化联邦学习的效果显著受连接设备网络拓扑结构的影响。通过建立一个简化的数值模型来研究这些系统的早期行为,我们提出了一种改进的人工神经网络初始化策略,该策略利用底层网络节点特征向量中心性的分布,从而显著提升了训练效率。此外,我们的研究探索了在所提出的初始化策略下的标度行为及环境参数的选择。这项工作为在分布式且无协调的环境中进行更高效、可扩展的人工神经网络训练铺平了道路,并深入揭示了网络结构与学习动态之间相互交织的作用。