Federated learning (FL) enables a loose set of participating clients to collaboratively learn a global model via coordination by a central server and with no need for data sharing. Existing FL approaches that rely on complex algorithms with massive models, such as deep neural networks (DNNs), suffer from computation and communication bottlenecks. In this paper, we first propose FedHDC, a federated learning framework based on hyperdimensional computing (HDC). FedHDC allows for fast and light-weight local training on clients, provides robust learning, and has smaller model communication overhead compared to learning with DNNs. However, current HDC algorithms get poor accuracy when classifying larger & more complex images, such as CIFAR10. To address this issue, we design FHDnn, which complements FedHDC with a self-supervised contrastive learning feature extractor. We avoid the transmission of the DNN and instead train only the HDC learner in a federated manner, which accelerates learning, reduces transmission cost, and utilizes the robustness of HDC to tackle network errors. We present a formal analysis of the algorithm and derive its convergence rate both theoretically, and show experimentally that FHDnn converges 3$\times$ faster vs. DNNs. The strategies we propose to improve the communication efficiency enable our design to reduce communication costs by 66$\times$ vs. DNNs, local client compute and energy consumption by ~1.5 - 6$\times$, while being highly robust to network errors. Finally, our proposed strategies for improving the communication efficiency have up to 32$\times$ lower communication costs with good accuracy.

翻译：联邦学习（FL）使一组松散参与的客户端能够通过中央服务器协调协作学习全局模型，且无需共享数据。现有依赖于复杂算法（如深度神经网络，DNN）的大规模模型的FL方法，面临计算与通信瓶颈。本文首先提出FedHDC——一种基于超维度计算（HDC）的联邦学习框架。与基于DNN的学习相比，FedHDC支持客户端快速轻量级本地训练、具有鲁棒学习能力，且模型通信开销更小。然而，当前HDC算法在分类CIFAR10等更大更复杂图像时精度较低。为解决此问题，我们设计了FHDnn，其为FedHDC补充了自监督对比学习特征提取器。我们避免传输DNN，仅以联邦方式训练HDC学习器，从而加速学习、降低传输成本，并利用HDC的鲁棒性应对网络错误。我们对算法进行了形式化分析，从理论上推导了其收敛速率，实验表明FHDnn收敛速度比DNN快3倍。我们提出的通信效率优化策略使设计相比DNN降低66倍通信成本，客户端本地计算与能耗降低约1.5-6倍，同时对网络错误具有高度鲁棒性。最后，我们提出的通信效率改进策略在保持良好精度的同时，通信成本降低达32倍。