Federated learning (FL) is a distributed learning paradigm that allows multiple decentralized clients to collaboratively learn a common model without sharing local data. Although local data is not exposed directly, privacy concerns nonetheless exist as clients' sensitive information can be inferred from intermediate computations. Moreover, such information leakage accumulates substantially over time as the same data is repeatedly used during the iterative learning process. As a result, it can be particularly difficult to balance the privacy-accuracy trade-off when designing privacy-preserving FL algorithms. This paper introduces Upcycled-FL, a simple yet effective strategy that applies first-order approximation at every even round of model update. Under this strategy, half of the FL updates incur no information leakage and require much less computational and transmission costs. We first conduct the theoretical analysis on the convergence (rate) of Upcycled-FL and then apply two perturbation mechanisms to preserve privacy. Extensive experiments on both synthetic and real-world data show that the Upcycled-FL strategy can be adapted to many existing FL frameworks and consistently improve the privacy-accuracy trade-off.

翻译：联邦学习（FL）是一种分布式学习范式，允许多个分散的客户端在不共享本地数据的情况下协作学习一个共同模型。尽管本地数据未直接暴露，但由于可以从中间计算中推断出客户端的敏感信息，隐私问题依然存在。此外，这种信息泄露会随着迭代学习过程中相同数据的重复使用而随时间显著累积。因此，在设计隐私保护的联邦学习算法时，平衡隐私与准确性的权衡尤为困难。本文提出Upcycled-FL，一种简单而有效的策略，在模型更新的每个偶数轮应用一阶近似。在该策略下，一半的联邦学习更新不会产生信息泄露，且计算与传输成本大幅降低。我们首先对Upcycled-FL的收敛性（速率）进行了理论分析，然后应用两种扰动机制来保护隐私。在合成数据与真实数据上的大量实验表明，Upcycled-FL策略可适配于许多现有联邦学习框架，并持续改善隐私与准确性的权衡。