Federated Learning (FL) allows machine learning models to train locally on individual mobile devices, synchronizing model updates via a shared server. This approach safeguards user privacy; however, it also generates a heterogeneous training environment due to the varying performance capabilities across devices. As a result, straggler devices with lower performance often dictate the overall training time in FL. In this work, we aim to alleviate this performance bottleneck due to stragglers by dynamically balancing the training load across the system. We introduce Invariant Dropout, a method that extracts a sub-model based on the weight update threshold, thereby minimizing potential impacts on accuracy. Building on this dropout technique, we develop an adaptive training framework, Federated Learning using Invariant Dropout (FLuID). FLuID offers a lightweight sub-model extraction to regulate computational intensity, thereby reducing the load on straggler devices without affecting model quality. Our method leverages neuron updates from non-straggler devices to construct a tailored sub-model for each straggler based on client performance profiling. Furthermore, FLuID can dynamically adapt to changes in stragglers as runtime conditions shift. We evaluate FLuID using five real-world mobile clients. The evaluations show that Invariant Dropout maintains baseline model efficiency while alleviating the performance bottleneck of stragglers through a dynamic, runtime approach.

翻译：联邦学习（FL）允许机器学习模型在单个移动设备上本地训练，并通过共享服务器同步模型更新。这种方法保护了用户隐私，但也因设备性能差异而产生异构训练环境。因此，性能较低的掉队设备往往决定FL的整体训练时间。本研究旨在通过动态平衡系统内训练负载来缓解掉队者导致的性能瓶颈。我们提出不变性Dropout方法，通过权重更新阈值提取子模型，从而最小化对准确率的潜在影响。基于该丢弃技术，我们开发了自适应训练框架——基于不变性Dropout的联邦学习（FLuID）。FLuID提供轻量级子模型提取以调节计算强度，在降低掉队设备负载的同时不影响模型质量。该方法利用非掉队设备的神经元更新，根据客户端性能画像为每个掉队设备构建定制子模型。此外，FLuID能根据运行时条件变化动态适应掉队者的状态。我们使用五台真实移动客户端评估FLuID，结果表明不变性Dropout能维持基线模型效率，并通过动态运行时方法缓解掉队者的性能瓶颈。