This paper considers the problem of distributed multi-agent learning, where the global aim is to minimize a sum of local objective (empirical loss) functions through local optimization and information exchange between neighbouring nodes. We introduce a Newton-type fully distributed optimization algorithm, Network-GIANT, which is based on GIANT, a Federated learning algorithm that relies on a centralized parameter server. The Network-GIANT algorithm is designed via a combination of gradient-tracking and a Newton-type iterative algorithm at each node with consensus based averaging of local gradient and Newton updates. We prove that our algorithm guarantees semi-global and exponential convergence to the exact solution over the network assuming strongly convex and smooth loss functions. We provide empirical evidence of the superior convergence performance of Network-GIANT over other state-of-art distributed learning algorithms such as Network-DANE and Newton-Raphson Consensus.

翻译：摘要：本文考虑分布式多智能体学习问题，其全局目标是通过局部优化和相邻节点间的信息交换，最小化一组局部目标（经验损失）函数之和。我们提出一种牛顿型完全分布式优化算法——网络化巨人算法（Network-GIANT），其基于依赖集中式参数服务器的联邦学习算法GIANT。Network-GIANT算法通过结合梯度追踪与各节点上的牛顿型迭代算法，并基于局部梯度和牛顿更新的共识平均而设计。我们证明，在强凸且光滑的损失函数假设下，该算法能保证在网络上实现半全局指数收敛至精确解。我们通过实验表明，Network-GIANT相较于Network-DANE和Newton-Raphson Consensus等当前最优分布式学习算法具有更优越的收敛性能。