With an increasing number of smart devices like internet of things (IoT) devices deployed in the field, offloadingtraining of neural networks (NNs) to a central server becomes more and more infeasible. Recent efforts toimprove users' privacy have led to on-device learning emerging as an alternative. However, a model trainedonly on a single device, using only local data, is unlikely to reach a high accuracy. Federated learning (FL)has been introduced as a solution, offering a privacy-preserving trade-off between communication overheadand model accuracy by sharing knowledge between devices but disclosing the devices' private data. Theapplicability and the benefit of applying baseline FL are, however, limited in many relevant use cases dueto the heterogeneity present in such environments. In this survey, we outline the heterogeneity challengesFL has to overcome to be widely applicable in real-world applications. We especially focus on the aspect ofcomputation heterogeneity among the participating devices and provide a comprehensive overview of recentworks on heterogeneity-aware FL. We discuss two groups: works that adapt the NN architecture and worksthat approach heterogeneity on a system level, covering Federated Averaging (FedAvg), distillation, and splitlearning-based approaches, as well as synchronous and asynchronous aggregation schemes.

翻译：随着物联网等智能设备在现实场景中的大规模部署，将神经网络训练任务集中到中央服务器的方式日益面临挑战。为提升用户隐私保护而催生的设备端学习方案虽可作为替代方案，但其依赖单设备本地数据训练的模型难以达到较高精度。联邦学习作为一种解决方案应运而生，通过设备间共享知识（但不泄露用户私有数据）实现通信开销与模型精度之间的隐私保护平衡。然而，在诸多实际应用场景中，由于设备环境的异构性，基础联邦学习的适用性及其性能优势均受到显著限制。本综述系统阐述了联邦学习在现实世界广泛部署前需克服的异构性挑战，重点剖析参与设备间的计算异构性特征，并全面梳理近期面向异构感知的联邦学习研究成果。我们将其分为两类方法：一类致力于调整神经网络架构，另一类从系统层面处理异构问题，涵盖联邦平均算法、知识蒸馏、基于拆分学习的方法及同步/异步聚合机制。