Spiking Neural Networks (SNNs) are biologically inspired alternatives to conventional Artificial Neural Networks (ANNs). Despite promising preliminary results, the trade-offs in the training of SNNs in a distributed scheme are not well understood. Here, we consider SNNs in a federated learning setting where a high-quality global model is created by aggregating multiple local models from the clients without sharing any data. We investigate federated learning for training multiple SNNs at clients when two mechanisms reduce the uplink communication cost: i) random masking of the model updates sent from the clients to the server; and ii) client dropouts where some clients do not send their updates to the server. We evaluated the performance of the SNNs using a subset of the Spiking Heidelberg digits (SHD) dataset. The results show that a trade-off between the random masking and the client drop probabilities is crucial to obtain a satisfactory performance for a fixed number of clients.
翻译:脉冲神经网络(SNN)是传统人工神经网络(ANN)的仿生替代方案。尽管初步结果令人鼓舞,但分布式方案下SNN训练的权衡机制仍不明确。本文探讨了联邦学习框架下的SNN模型——该框架通过聚合客户端本地模型构建高质量全局模型,且无需共享任何数据。我们研究了当两种机制降低上行通信成本时,在客户端训练多个SNN的联邦学习方法:i) 对客户端发送至服务器的模型更新进行随机掩码;ii) 部分客户端不向服务器发送更新的客户端丢弃策略。我们使用Spiking Heidelberg Digits(SHD)数据集的子集评估了SNN性能。结果表明:在固定客户端数量下,随机掩码与客户端丢弃概率之间的权衡对获得满意性能至关重要。