Decentralized federated learning (DFL), a serverless variant of federated learning, poses unique challenges for parameter-efficient fine-tuning due to the factorized structure of low-rank adaptation (LoRA). Unlike linear parameters, decentralized aggregation of LoRA updates introduces topology-dependent cross terms that can destabilize training under dynamic communication graphs. We propose \texttt{TAD-LoRA}, a Topology-Aware Decentralized Low-Rank Adaptation framework that coordinates the updates and mixing of LoRA factors to control inter-client misalignment. We theoretically prove the convergence of \texttt{TAD-LoRA} under non-convex objectives, explicitly characterizing the trade-off between topology-induced cross-term error and block-coordinate representation bias governed by the switching interval of alternative training. Experiments under various communication conditions validate our analysis, showing that \texttt{TAD-LoRA} achieves robust performance across different communication scenarios, remaining competitive in strongly connected topologies and delivering clear gains under moderately and weakly connected topologies, with particularly strong results on the MNLI dataset.

翻译：分散式联邦学习作为联邦学习的无服务器变体，由于低秩适配的参数分解结构，在参数高效微调方面面临独特挑战。与线性参数不同，LoRA更新的分散式聚合会引入依赖于拓扑结构的交叉项，这些项在动态通信图下可能破坏训练稳定性。我们提出\texttt{TAD-LoRA}——一种拓扑感知的分散式低秩适配框架，通过协调LoRA因子的更新与混合来控制客户端间的错位问题。我们在非凸目标下从理论上证明了\texttt{TAD-LoRA}的收敛性，明确刻画了由交替训练切换间隔控制的拓扑诱导交叉项误差与块坐标表示偏差之间的权衡关系。多种通信条件下的实验验证了我们的分析，表明\texttt{TAD-LoRA}在不同通信场景中均能实现鲁棒性能：在强连通拓扑中保持竞争力，在中等及弱连通拓扑中取得显著增益，尤其在MNLI数据集上表现出色。