In this paper, we study the (decentralized) distributed optimization problem with high-dimensional sparse structure. Building upon the FedDA algorithm, we propose a (Decentralized) FedDA-GT algorithm, which combines the \textbf{gradient tracking} technique. It is able to eliminate the heterogeneity among different clients' objective functions while ensuring a dimension-free convergence rate. Compared to the vanilla FedDA approach, (D)FedDA-GT can significantly reduce the communication complexity, from ${O}(s^2\log d/\varepsilon^{3/2})$ to a more efficient ${O}(s^2\log d/\varepsilon)$. In cases where strong convexity is applicable, we introduce a multistep mechanism resulting in the Multistep ReFedDA-GT algorithm, a minor modified version of FedDA-GT. This approach achieves an impressive communication complexity of ${O}\left(s\log d \log \frac{1}{\varepsilon}\right)$ through repeated calls to the ReFedDA-GT algorithm. Finally, we conduct numerical experiments, illustrating that our proposed algorithms enjoy the dual advantage of being dimension-free and heterogeneity-free.

翻译：本文研究了具有高维稀疏结构的（去中心化）分布式优化问题。基于FedDA算法，我们提出了（去中心化）FedDA-GT算法，该算法融合了**梯度追踪**技术。该算法能够消除不同客户端目标函数之间的异质性，同时确保收敛速率与维度无关。与原始FedDA方法相比，(D)FedDA-GT可显著降低通信复杂度，从${O}(s^2\log d/\varepsilon^{3/2})$提升至更高效的${O}(s^2\log d/\varepsilon)$。在适用强凸性的情形下，我们引入了多步机制，从而得到FedDA-GT的微调版本——多步ReFedDA-GT算法。通过重复调用ReFedDA-GT算法，该方法实现了${O}\left(s\log d \log \frac{1}{\varepsilon}\right)$的优异通信复杂度。最后，我们进行了数值实验，结果表明所提出的算法兼具维度无关性与异质性无关性的双重优势。