We present P2PL, a practical multi-device peer-to-peer deep learning algorithm that, unlike the federated learning paradigm, does not require coordination from edge servers or the cloud. This makes P2PL well-suited for the sheer scale of beyond-5G computing environments like smart cities that otherwise create range, latency, bandwidth, and single point of failure issues for federated approaches. P2PL introduces max norm synchronization to catalyze training, retains on-device deep model training to preserve privacy, and leverages local inter-device communication to implement distributed consensus. Each device iteratively alternates between two phases: 1) on-device learning and 2) distributed cooperation where they combine model parameters with nearby devices. We empirically show that all participating devices achieve the same test performance attained by federated and centralized training -- even with 100 devices and relaxed singly stochastic consensus weights. We extend these experimental results to settings with diverse network topologies, sparse and intermittent communication, and non-IID data distributions.

翻译：我们提出P2PL，一种实用的多设备端到端深度学习算法。与联邦学习范式不同，该算法无需边缘服务器或云端的协调，因此非常适合超5G计算环境（如智慧城市）的庞大规模——联邦方法在此类场景中会面临范围、延迟、带宽及单点故障等问题。P2PL引入最大范数同步机制以加速训练，保留设备端深度模型训练以保护隐私，并利用本地设备间通信实现分布式共识。每台设备迭代交替执行两个阶段：1）设备端学习，2）分布式协作——在此阶段设备与邻近设备合并模型参数。实验表明，即使采用100台设备并使用松弛单随机共识权重，所有参与设备均能达到与联邦训练及集中式训练相同的测试性能。我们将这些实验结果扩展至多样化网络拓扑、间歇性稀疏通信及非独立同分布数据分布等场景。