The future of machine learning lies in moving data collection along with training to the edge. Federated Learning, for short FL, has been recently proposed to achieve this goal. The principle of this approach is to aggregate models learned over a large number of distributed clients, i.e., resource-constrained mobile devices that collect data from their environment, to obtain a new more general model. The latter is subsequently redistributed to clients for further training. A key feature that distinguishes federated learning from data-center-based distributed training is the inherent heterogeneity. In this work, we introduce and analyse a novel aggregation framework that allows for formalizing and tackling computational heterogeneity in federated optimization, in terms of both heterogeneous data and local updates. Proposed aggregation algorithms are extensively analyzed from a theoretical, and an experimental prospective.

翻译：机器学习的未来在于将数据收集与训练迁移至边缘端。联邦学习（Federated Learning，简称FL）近期被提出以实现这一目标。该方法的原理是通过聚合分布在大量分布式客户端（即从环境中收集数据的资源受限移动设备）上学习到的模型，获得一个更通用的新模型，随后将该模型重新分发给各客户端进行进一步训练。联邦学习与数据中心式分布式训练的关键区别在于其固有的异质性。本文介绍并分析了一种新型聚合框架，该框架能够从异构数据和局部更新两方面形式化并应对联邦优化中的计算异质性。我们提出的聚合算法从理论和实验两个角度得到了全面分析。