The widespread adoption of Federated Learning (FL), a privacy-preserving distributed learning methodology, has been impeded by the challenge of high communication overheads, typically arising from the transmission of large-scale models. Existing adaptive quantization methods, designed to mitigate these overheads, operate under the impractical assumption of uniform device participation in every training round. Additionally, these methods are limited in their adaptability due to the necessity of manual quantization level selection and often overlook biases inherent in local devices' data, thereby affecting the robustness of the global model. In response, this paper introduces AQUILA (adaptive quantization of lazily-aggregated gradients), a novel adaptive framework devised to effectively handle these issues, enhancing the efficiency and robustness of FL. AQUILA integrates a sophisticated device selection method that prioritizes the quality and usefulness of device updates. Utilizing the exact global model stored by devices, it enables a more precise device selection criterion, reduces model deviation, and limits the need for hyperparameter adjustments. Furthermore, AQUILA presents an innovative quantization criterion, optimized to improve communication efficiency while assuring model convergence. Our experiments demonstrate that AQUILA significantly decreases communication costs compared to existing methods, while maintaining comparable model performance across diverse non-homogeneous FL settings, such as Non-IID data and heterogeneous model architectures.

翻译：联邦学习（FL）作为一种隐私保护的分布式学习方法，其广泛采用因大规模模型传输带来的高通信开销而受到阻碍。现有的自适应量化方法旨在缓解这些开销，但均基于每轮训练中所有设备均匀参与的不可实际假设。此外，这些方法因需手动选择量化等级而适应性有限，且常忽略本地设备数据固有的偏差，从而影响全局模型的鲁棒性。为此，本文提出AQUILA（惰性聚合梯度的自适应量化），一种新颖的自适应框架，旨在有效处理这些问题，提升FL的效率与鲁棒性。AQUILA集成了精密的设备选择方法，优先考虑设备更新的质量与实用性。通过利用设备存储的精确全局模型，它实现了更精确的设备选择标准，减少了模型偏差，并限制了超参数调整的需求。此外，AQUILA提出了一种创新的量化标准，旨在提升通信效率的同时确保模型收敛。实验表明，与现有方法相比，AQUILA在多种非均匀FL设置（如非独立同分布数据与异构模型架构）下，显著降低了通信成本，同时保持了可比的模型性能。