Federated learning (FL) is usually performed on resource-constrained edge devices, e.g., with limited memory for the computation. If the required memory to train a model exceeds this limit, the device will be excluded from the training. This can lead to a lower accuracy as valuable data and computation resources are excluded from training, also causing bias and unfairness. The FL training process should be adjusted to such constraints. The state-of-the-art techniques propose training subsets of the FL model at constrained devices, reducing their resource requirements for training. But these techniques largely limit the co-adaptation among parameters of the model and are highly inefficient, as we show: it is actually better to train a smaller (less accurate) model by the system where all the devices can train the model end-to-end, than applying such techniques. We propose a new method that enables successive freezing and training of the parameters of the FL model at devices, reducing the training's resource requirements at the devices, while still allowing enough co-adaptation between parameters. We show through extensive experimental evaluation that our technique greatly improves the accuracy of the trained model (by 52.4 p.p.) compared with the state of the art, efficiently aggregating the computation capacity available on distributed devices.

翻译：联邦学习（FL）通常在资源受限的边缘设备上执行，例如计算内存有限。若训练模型所需内存超出此限制，该设备将被排除在训练之外。这可能导致精度降低，因为宝贵的数据和计算资源被排除在训练之外，同时引发偏差与不公平。FL训练过程应适配此类约束。现有技术提出在受限设备上训练FL模型的子集，以降低其训练资源需求。但我们证明：这些技术严重限制了模型参数间的协同适应，且效率极低——实际上，若所有设备都能端到端训练模型，采用系统训练一个更小（精度较低）的模型反而优于应用此类技术。我们提出一种新方法，可在设备上对FL模型参数进行逐次冻结与训练，在降低设备训练资源需求的同时，允许参数间充分的协同适应。通过大量实验评估表明，与现有技术相比，我们的方法将训练模型的精度显著提升52.4个百分点，高效聚合了分布式设备的可用计算能力。