Efficiently running federated learning (FL) on resource-constrained devices is challenging since they are required to train computationally intensive deep neural networks (DNN) independently. DNN partitioning-based FL (DPFL) has been proposed as one mechanism to accelerate training where the layers of a DNN (or computation) are offloaded from the device to the server. However, this creates significant communication overheads since the activation and gradient need to be transferred between the device and the server during training. While current research reduces the communication introduced by DNN partitioning using local loss-based methods, we demonstrate that these methods are ineffective in improving the overall efficiency (communication overhead and training speed) of a DPFL system. This is because they suffer from accuracy degradation and ignore the communication costs incurred when transferring the activation from the device to the server. This paper proposes EcoFed - a communication efficient framework for DPFL systems. EcoFed eliminates the transmission of the gradient by developing pre-trained initialization of the DNN model on the device for the first time. This reduces the accuracy degradation seen in local loss-based methods. In addition, EcoFed proposes a novel replay buffer mechanism and implements a quantization-based compression technique to reduce the transmission of the activation. It is experimentally demonstrated that EcoFed can significantly reduce the communication cost by up to 114x and accelerates training by up to 25.66x when compared to classic FL. Compared to vanilla DPFL, EcoFed achieves a 13.78x communication reduction and 2.83x training speed up.

翻译：在资源受限设备上高效运行联邦学习（FL）是一项挑战，因为这类设备需独立训练计算密集的深度神经网络（DNN）。基于DNN分割的联邦学习（DPFL）被提出作为一种加速训练机制，其将DNN的层（或计算）从设备卸载至服务器。然而，训练过程中激活值与梯度需要在设备与服务器间传输，导致显著的通信开销。尽管现有研究通过基于局部损失的方法来减少DNN分割引入的通信量，但本文证明这些方法在提升DPFL系统整体效率（通信开销与训练速度）方面效果有限，因其存在精度下降问题，且忽视了激活值从设备传输至服务器所需的通信成本。本文提出EcoFed——一种面向DPFL系统的高效通信框架。EcoFed通过首次在设备端开发DNN模型的预训练初始化，消除了梯度传输需求，从而缓解了基于局部损失方法的精度退化问题。此外，EcoFed提出新型重放缓冲区机制，并实现基于量化的压缩技术以减少激活值传输。实验表明，与经典FL相比，EcoFed可降低高达114倍的通信成本，并将训练加速至25.66倍；与原始DPFL相比，EcoFed实现13.78倍的通信压缩与2.83倍的训练加速。