Federated learning (FL) is a general principle for decentralized clients to train a server model collectively without sharing local data. FL is a promising framework with practical applications, but its standard training paradigm requires the clients to backpropagate through the model to compute gradients. Since these clients are typically edge devices and not fully trusted, executing backpropagation on them incurs computational and storage overhead as well as white-box vulnerability. In light of this, we develop backpropagation-free federated learning, dubbed BAFFLE, in which backpropagation is replaced by multiple forward processes to estimate gradients. BAFFLE is 1) memory-efficient and easily fits uploading bandwidth; 2) compatible with inference-only hardware optimization and model quantization or pruning; and 3) well-suited to trusted execution environments, because the clients in BAFFLE only execute forward propagation and return a set of scalars to the server. Empirically we use BAFFLE to train deep models from scratch or to finetune pretrained models, achieving acceptable results. Code is available in https://github.com/FengHZ/BAFFLE.

翻译：联邦学习是一种通用原则，使分散的客户端无需共享本地数据即可共同训练服务器模型。联邦学习是一个具有实际应用前景的框架，但其标准训练范式要求客户端通过模型进行反向传播以计算梯度。由于这些客户端通常是边缘设备且不完全可信，在其上执行反向传播会带来计算和存储开销，以及白盒脆弱性。鉴于此，我们开发出无需反向传播的联邦学习，称为BAFFLE，其中反向传播被多次前向传播替代以估计梯度。BAFFLE具有以下特点：1）内存高效且易于匹配上传带宽；2）兼容仅需推理的硬件优化、模型量化或剪枝；3）特别适用于可信执行环境，因为BAFFLE中的客户端仅执行前向传播并将一组标量返回给服务器。实验中，我们使用BAFFLE从头训练深度模型或微调预训练模型，取得了可接受的结果。代码可在https://github.com/FengHZ/BAFFLE获取。