Federated Learning (FL) aggregates locally trained models from individual clients to construct a global model. While FL enables learning a model with data privacy, it often suffers from significant performance degradation when client data distributions are heterogeneous. Many previous FL algorithms have addressed this issue by introducing various proximal restrictions. These restrictions aim to encourage global alignment by constraining the deviation of local learning from the global objective. However, they inherently limit local learning by interfering with the original local objectives. Recently, an alternative approach has emerged to improve local learning generality. By obtaining local models within a smooth loss landscape, this approach mitigates conflicts among different local objectives of the clients. Yet, it does not ensure stable global alignment, as local learning does not take the global objective into account. In this study, we propose Federated Stability on Learning (FedSoL), which combines both the concepts of global alignment and local generality. In FedSoL, the local learning seeks a parameter region robust against proximal perturbations. This strategy introduces an implicit proximal restriction effect in local learning while maintaining the original local objective for parameter update. Our experiments show that FedSoL consistently achieves state-of-the-art performance on various setups.

翻译：联邦学习（Federated Learning, FL）通过聚合各客户端的本地训练模型来构建全局模型。尽管FL在数据隐私保护下实现了模型学习，但当客户端数据分布呈现异质性时，它常常遭受显著的性能退化。许多先前的FL算法通过引入各类近端约束来解决此问题。这些约束旨在通过限制本地学习偏离全局目标来促进全局对齐。然而，它们通过干扰原始本地目标而本质上限制了本地学习。近期，一种提升本地学习泛化性的替代方案出现：通过在平滑的损失景观中获得本地模型，该方法缓解了客户端间不同本地目标之间的冲突。但该方法并未确保稳定的全局对齐，因为本地学习未考虑全局目标。在本研究中，我们提出了"联邦学习稳定性（FedSoL）"，它结合了全局对齐与局部泛化这两种理念。在FedSoL中，本地学习会寻找一个对近端扰动具有鲁棒性的参数区域。该策略在本地学习中引入隐式的近端约束效果，同时保留用于参数更新的原始本地目标。实验表明，FedSoL在各种设置下均能持续实现最先进的性能。