We present P2PL, a practical multi-device peer-to-peer deep learning algorithm that, unlike the federated learning paradigm, does not require coordination from edge servers or the cloud. This makes P2PL well-suited for the sheer scale of beyond-5G computing environments like smart cities that otherwise create range, latency, bandwidth, and single point of failure issues for federated approaches. P2PL introduces max norm synchronization to catalyze training, retains on-device deep model training to preserve privacy, and leverages local inter-device communication to implement distributed consensus. Each device iteratively alternates between two phases: 1) on-device learning and 2) peer-to-peer cooperation where they combine model parameters with nearby devices. We empirically show that all participating devices achieve the same test performance attained by federated and centralized training -- even with 100 devices and relaxed singly stochastic consensus weights. We extend these experimental results to settings with diverse network topologies, sparse and intermittent communication, and non-IID data distributions.

翻译：本文提出P2PL，一种实用的多设备对等深度学习算法。与联邦学习范式不同，该算法无需边缘服务器或云端的协调，使其特别适用于后5G计算环境（如智慧城市）的庞大规模——而联邦学习方法在此类场景中会面临范围、延迟、带宽及单点故障等问题。P2PL引入最大范数同步以加速训练，保留设备端深度模型训练以保护隐私，并利用本地设备间通信实现分布式共识。每台设备迭代交替执行两个阶段：1）设备端学习，2）与邻近设备合并模型参数的对等协作。实验表明，所有参与设备均能达到与联邦训练和集中式训练相同的测试性能——即便在100台设备及松弛单随机共识权重条件下。我们将实验结果扩展到多种网络拓扑、稀疏间歇通信及非独立同分布数据分布场景。