Federated learning (FL) is usually performed on resource-constrained edge devices, e.g., with limited memory for the computation. If the required memory to train a model exceeds this limit, the device will be excluded from the training. This can lead to a lower accuracy as valuable data and computation resources are excluded from training, also causing bias and unfairness. The FL training process should be adjusted to such constraints. The state-of-the-art techniques propose training subsets of the FL model at constrained devices, reducing their resource requirements for training. But these techniques largely limit the co-adaptation among parameters of the model and are highly inefficient, as we show: it is actually better to train a smaller (less accurate) model by the system where all the devices can train the model end-to-end, than applying such techniques. We propose a new method that enables successive freezing and training of the parameters of the FL model at devices, reducing the training's resource requirements at the devices, while still allowing enough co-adaptation between parameters. We show through extensive experimental evaluation that our technique greatly improves the accuracy of the trained model (by 52.4 p.p.) compared with the state of the art, efficiently aggregating the computation capacity available on distributed devices.

翻译：联邦学习（FL）通常在资源受限的边缘设备上执行，例如计算内存有限。若训练模型所需的内存超出此限制，设备将被排除在训练过程之外。这可能导致准确率下降，因为宝贵的数据和计算资源被排除在训练之外，同时引发偏差与不公平问题。FL训练过程应针对此类约束进行调整。现有先进技术提出在受限设备上训练FL模型的子集，以降低其训练资源需求。但我们证明，这些技术严重限制了模型参数间的协同适应能力，且效率极低：实际上，由所有设备都能端到端训练模型的系统训练一个规模更小（精度更低）的模型，效果反而优于采用此类技术。我们提出一种新方法，在设备上对FL模型参数进行逐次冻结与训练，既降低设备端的训练资源需求，又保留参数间足够的协同适应能力。通过大量实验评估表明，与现有先进技术相比，我们的技术可将训练模型的准确率大幅提升（52.4个百分点），有效聚合分布式设备上的可用计算容量。