Federated learning (FL) is a general principle for decentralized clients to train a server model collectively without sharing local data. FL is a promising framework with practical applications, but its standard training paradigm requires the clients to backpropagate through the model to compute gradients. Since these clients are typically edge devices and not fully trusted, executing backpropagation on them incurs computational and storage overhead as well as white-box vulnerability. In light of this, we develop backpropagation-free federated learning, dubbed BAFFLE, in which backpropagation is replaced by multiple forward processes to estimate gradients. BAFFLE is 1) memory-efficient and easily fits uploading bandwidth; 2) compatible with inference-only hardware optimization and model quantization or pruning; and 3) well-suited to trusted execution environments, because the clients in BAFFLE only execute forward propagation and return a set of scalars to the server. Empirically we use BAFFLE to train deep models from scratch or to finetune pretrained models, achieving acceptable results. Code is available in https://github.com/FengHZ/BAFFLE.

翻译：联邦学习是一种通用原则，使得去中心化客户端能够在无需共享本地数据的情况下共同训练服务器模型。联邦学习作为一个具有实际应用前景的框架，但其标准训练范式要求客户端通过模型进行反向传播以计算梯度。由于这些客户端通常是边缘设备且不完全可信，执行反向传播会带来计算和存储开销，并存在白盒攻击风险。鉴于此，我们开发了无需反向传播的联邦学习方案，命名为BAFFLE，该方法通过多次前向过程替代反向传播来估计梯度。BAFFLE具有以下特性：1）内存高效且易于匹配上传带宽；2）兼容仅推理硬件优化、模型量化或剪枝；3）特别适用于可信执行环境，因为BAFFLE中的客户端仅执行前向传播并向服务器返回一组标量值。实验表明，我们使用BAFFLE从零训练深度模型或微调预训练模型均能取得可接受的结果。代码已开源至 https://github.com/FengHZ/BAFFLE。