Edge networks call for communication efficient (low overhead) and robust distributed optimization (DO) algorithms. These are, in fact, desirable qualities for DO frameworks, such as federated edge learning techniques, in the presence of data and system heterogeneity, and in scenarios where internode communication is the main bottleneck. Although computationally demanding, Newton-type (NT) methods have been recently advocated as enablers of robust convergence rates in challenging DO problems where edge devices have sufficient computational power. Along these lines, in this work we propose Q-SHED, an original NT algorithm for DO featuring a novel bit-allocation scheme based on incremental Hessian eigenvectors quantization. The proposed technique is integrated with the recent SHED algorithm, from which it inherits appealing features like the small number of required Hessian computations, while being bandwidth-versatile at a bit-resolution level. Our empirical evaluation against competing approaches shows that Q-SHED can reduce by up to 60% the number of communication rounds required for convergence.

翻译：边缘网络要求通信高效（低开销）且鲁棒的分布式优化算法。这些特性对于联邦边缘学习等分布式优化框架而言，在存在数据与系统异构性、且节点间通信为主要瓶颈的场景中尤为关键。尽管计算成本高昂，牛顿类方法近期被倡导作为关键解决方案，能够在边缘设备具备足够算力的挑战性分布式优化问题中实现稳健收敛。基于此，本文提出Q-SHED——一种原创的牛顿类分布式优化算法，其核心包含基于增量式Hessian特征向量量化的新型比特分配方案。该技术继承自近期SHED算法，沿袭其低Hessian计算量的优势特性，同时具备比特级分辨率的带宽灵活性。实验评估表明，与对比方法相比，Q-SHED可减少最高60%的收敛所需通信轮次。