Classical paradigms for distributed learning, such as federated or decentralized gradient descent, employ consensus mechanisms to enforce homogeneity among agents. While these strategies have proven effective in i.i.d. scenarios, they can result in significant performance degradation when agents follow heterogeneous objectives or data. Distributed strategies for multitask learning, on the other hand, induce relationships between agents in a more nuanced manner, and encourage collaboration without enforcing consensus. We develop a generalization of the exact diffusion algorithm for subspace constrained multitask learning over networks, and derive an accurate expression for its mean-squared deviation when utilizing noisy gradient approximations. We verify numerically the accuracy of the predicted performance expressions, as well as the improved performance of the proposed approach over alternatives based on approximate projections.

翻译：分布式学习的经典范式，例如联邦或去中心化梯度下降，采用共识机制来强制智能体之间的同质性。尽管这些策略在独立同分布场景中已被证明有效，但当智能体遵循异质性目标或数据时，它们可能导致显著的性能下降。另一方面，针对多任务学习的分布式策略以更细致的方式建立智能体间的关系，并鼓励协作而不强制达成共识。我们针对网络上子空间约束的多任务学习，提出了一种精确扩散算法的推广形式，并推导了其在利用含噪梯度近似时的均方偏差的精确表达式。我们通过数值实验验证了所预测性能表达式的准确性，以及所提方法相比基于近似投影的替代方案的性能提升。