Federated learning faces a critical challenge in balancing communication efficiency with rapid convergence, especially for second-order methods. While Newton-type algorithms achieve linear convergence in communication rounds, transmitting full Hessian matrices is often impractical due to quadratic complexity. We introduce Federated Learning with Enhanced Nesterov-Newton Sketch (FLeNS), a novel method that harnesses both the acceleration capabilities of Nesterov's method and the dimensionality reduction benefits of Hessian sketching. FLeNS approximates the centralized Newton's method without relying on the exact Hessian, significantly reducing communication overhead. By combining Nesterov's acceleration with adaptive Hessian sketching, FLeNS preserves crucial second-order information while preserving the rapid convergence characteristics. Our theoretical analysis, grounded in statistical learning, demonstrates that FLeNS achieves super-linear convergence rates in communication rounds - a notable advancement in federated optimization. We provide rigorous convergence guarantees and characterize tradeoffs between acceleration, sketch size, and convergence speed. Extensive empirical evaluation validates our theoretical findings, showcasing FLeNS's state-of-the-art performance with reduced communication requirements, particularly in privacy-sensitive and edge-computing scenarios. The code is available at https://github.com/sunnyinAI/FLeNS

翻译：联邦学习面临着一个关键挑战：如何在通信效率与快速收敛之间取得平衡，这对于二阶方法尤为突出。虽然牛顿类算法在通信轮次中能够实现线性收敛，但由于二次复杂度，传输完整的Hessian矩阵通常不切实际。本文提出了一种基于增强型Nesterov-牛顿草图方法的联邦学习（FLeNS），该方法同时利用了Nesterov方法的加速能力和Hessian草图技术的降维优势。FLeNS无需依赖精确的Hessian矩阵即可逼近中心化牛顿方法，从而显著降低了通信开销。通过将Nesterov加速与自适应Hessian草图技术相结合，FLeNS在保持快速收敛特性的同时保留了关键的二阶信息。基于统计学习的理论分析表明，FLeNS在通信轮次中实现了超线性收敛速率——这是联邦优化领域的重要进展。我们提供了严格的收敛性保证，并刻画了加速机制、草图尺寸与收敛速度之间的权衡关系。大量实证评估验证了我们的理论发现，展示了FLeNS在降低通信需求方面的先进性能，尤其在隐私敏感和边缘计算场景中表现突出。代码已发布于 https://github.com/sunnyinAI/FLeNS