Federated Learning (FL) allows machine learning models to train locally on individual mobile devices, synchronizing model updates via a shared server. This approach safeguards user privacy; however, it also generates a heterogeneous training environment due to the varying performance capabilities across devices. As a result, straggler devices with lower performance often dictate the overall training time in FL. In this work, we aim to alleviate this performance bottleneck due to stragglers by dynamically balancing the training load across the system. We introduce Invariant Dropout, a method that extracts a sub-model based on the weight update threshold, thereby minimizing potential impacts on accuracy. Building on this dropout technique, we develop an adaptive training framework, Federated Learning using Invariant Dropout (FLuID). FLuID offers a lightweight sub-model extraction to regulate computational intensity, thereby reducing the load on straggler devices without affecting model quality. Our method leverages neuron updates from non-straggler devices to construct a tailored sub-model for each straggler based on client performance profiling. Furthermore, FLuID can dynamically adapt to changes in stragglers as runtime conditions shift. We evaluate FLuID using five real-world mobile clients. The evaluations show that Invariant Dropout maintains baseline model efficiency while alleviating the performance bottleneck of stragglers through a dynamic, runtime approach.

翻译：联邦学习（FL）允许机器学习模型在个人移动设备上本地训练，并通过共享服务器同步模型更新。该方法保护了用户隐私，但也因设备性能差异产生异构训练环境。因此，性能较低的落后者设备常决定FL的整体训练时间。本研究旨在通过动态平衡系统训练负载，缓解落后者导致的性能瓶颈。我们提出不变性丢弃法（Invariant Dropout），该方法基于权重更新阈值提取子模型，从而最小化对准确率的潜在影响。基于该丢弃技术，我们开发了自适应训练框架——基于不变性丢弃法的联邦学习（FLuID）。FLuID提供轻量级子模型提取以调控计算强度，从而在不影响模型质量的前提下减轻落后者设备的负载。该方法利用非落后者设备的神经元更新，基于客户端性能画像为每个落后者构建定制子模型。此外，FLuID可根据运行时条件变化动态适应落后者变化。我们使用五组真实移动客户端评估FLuID，结果表明不变性丢弃法在维持基线模型效率的同时，通过动态运行时方法缓解了落后者的性能瓶颈。