Devices participating in federated learning (FL) typically have heterogeneous communication, computation, and memory resources. However, in synchronous FL, all devices need to finish training by the same deadline dictated by the server. Our results show that training a smaller subset of the neural network (NN) at constrained devices, i.e., dropping neurons/filters as proposed by state of the art, is inefficient, preventing these devices to make an effective contribution to the model. This causes unfairness w.r.t the achievable accuracies of constrained devices, especially in cases with a skewed distribution of class labels across devices. We present a novel FL technique, CoCoFL, which maintains the full NN structure on all devices. To adapt to the devices' heterogeneous resources, CoCoFL freezes and quantizes selected layers, reducing communication, computation, and memory requirements, whereas other layers are still trained in full precision, enabling to reach a high accuracy. Thereby, CoCoFL efficiently utilizes the available resources on devices and allows constrained devices to make a significant contribution to the FL system, increasing fairness among participants (accuracy parity) and significantly improving the final accuracy of the model.

翻译：参与联邦学习（FL）的设备通常具有异构的通信、计算和内存资源。然而在同步联邦学习中，所有设备需在服务器规定的统一截止时间前完成训练。我们的研究表明，在资源受限设备上训练神经网络的子集（即按照现有技术方案丢弃神经元/滤波器）效率低下，阻碍了这些设备对模型做出有效贡献。这导致受限设备在可达到的精度方面存在不公平性，尤其在设备间类别标签分布偏斜的情况下更为严重。我们提出一种新型联邦学习技术CoCoFL，该技术在所有设备上保持完整的神经网络结构。为适应设备的异构资源，CoCoFL冻结并量化选定层，降低通信、计算和内存需求，而其他层仍以全精度训练，从而能够达到高精度。由此，CoCoFL高效利用设备可用资源，使受限设备能为联邦学习系统做出显著贡献，提升参与者间的公平性（精度均衡性），并显著改善模型的最终精度。