The biological neural network is a vast and diverse structure with high neural heterogeneity. Conventional Artificial Neural Networks (ANNs) primarily focus on modifying the weights of connections through training while modeling neurons as highly homogenized entities and lacking exploration of neural heterogeneity. Only a few studies have addressed neural heterogeneity by optimizing neuronal properties and connection weights to ensure network performance. However, this strategy impact the specific contribution of neuronal heterogeneity. In this paper, we first demonstrate the challenges faced by backpropagation-based methods in optimizing Spiking Neural Networks (SNNs) and achieve more robust optimization of heterogeneous neurons in random networks using an Evolutionary Strategy (ES). Experiments on tasks such as working memory, continuous control, and image recognition show that neuronal heterogeneity can improve performance, particularly in long sequence tasks. Moreover, we find that membrane time constants play a crucial role in neural heterogeneity, and their distribution is similar to that observed in biological experiments. Therefore, we believe that the neglected neuronal heterogeneity plays an essential role, providing new approaches for exploring neural heterogeneity in biology and new ways for designing more biologically plausible neural networks.

翻译：生物神经网络是一个高度异质化的庞大结构。传统人工神经网络主要通过训练调整连接权重，将神经元建模为高度同质化的实体，缺乏对神经异质性的探索。仅有少数研究通过优化神经元属性与连接权重来保证网络性能，从而涉及神经异质性。然而，这种策略影响了神经异质性的具体贡献。本文首先展示了基于反向传播的方法在优化脉冲神经网络（SNNs）时面临的挑战，并利用进化策略（ES）在随机网络中实现了对异质性神经元的更鲁棒优化。在诸如工作记忆、连续控制和图像识别等任务上的实验表明，神经异质性能够提升性能，尤其在长序列任务中效果显著。此外，我们发现膜时间常数在神经异质性中起着关键作用，其分布与生物学实验中观察到的分布相似。因此，我们认为被忽视的神经异质性发挥着至关重要的作用，这为探索生物学中的神经异质性提供了新方法，也为设计更具生物合理性的神经网络开辟了新途径。