Federated Learning (FL) revolutionizes collaborative machine learning among Internet of Things (IoT) devices by enabling them to train models collectively while preserving data privacy. FL algorithms fall into two primary categories: synchronous and asynchronous. While synchronous FL efficiently handles straggler devices, it can compromise convergence speed and model accuracy. In contrast, asynchronous FL allows all devices to participate but incurs high communication overhead and potential model staleness. To overcome these limitations, the semi-synchronous FL framework introduces client tiering based on computing and communication latencies. Clients in different tiers upload their local models at distinct frequencies, striking a balance between straggler mitigation and communication costs. Enter the DecantFed algorithm (Dynamic client clustering, bandwidth allocation, and local training for semi-synchronous Federated learning), a dynamic solution that optimizes client clustering, bandwidth allocation, and local training workloads to maximize data sample processing rates. Additionally, DecantFed adapts client learning rates according to their tiers, addressing the model staleness problem. The algorithm's performance shines in extensive simulations using benchmark datasets, including MNIST and CIFAR-10, under independent and identically distributed (IID) and non-IID scenarios. DecantFed outpaces FedAvg and FedProx in terms of convergence speed and delivers a remarkable minimum 28% boost in model accuracy compared to FedProx.
翻译:联邦学习(FL)通过使物联网设备在保护数据隐私的同时协同训练模型,彻底改变了物联网设备间的协作机器学习方式。FL算法主要分为同步与异步两类。虽然同步FL能高效处理掉队设备,但会牺牲收敛速度与模型精度;而异步FL虽允许所有设备参与,却带来高通信开销与模型陈旧性问题。为克服这些局限,半同步FL框架引入了基于计算与通信延迟的客户端分层机制。不同层的客户端以不同频率上传本地模型,在缓解掉队问题与降低通信成本之间取得平衡。本文提出DecantFed算法(面向半同步联邦学习的动态客户端聚类、带宽分配与本地训练优化),这是一种动态解决方案,通过优化客户端聚类、带宽分配和本地训练工作负载来最大化数据样本处理速率。此外,DecantFed根据客户端层别自适应调整其学习率,从而解决模型陈旧性问题。在基准数据集(包括MNIST和CIFAR-10)的独立同分布(IID)与非独立同分布(non-IID)场景下进行的广泛仿真中,该算法性能表现优异。与FedAvg和FedProx相比,DecantFed在收敛速度方面更具优势,且相较于FedProx,模型精度至少提升28%。