Traditional large-scale neuroscience models and machine learning utilize simplified models of individual neurons, relying on collective activity and properly adjusted connections to perform complex computations. However, each biological cortical neuron is inherently a sophisticated computational device, as corroborated in a recent study where it took a deep artificial neural network with millions of parameters to replicate the input-output relationship of a detailed biophysical model of a cortical pyramidal neuron. We question the necessity for these many parameters and introduce the Expressive Leaky Memory (ELM) neuron, a biologically inspired, computationally expressive, yet efficient model of a cortical neuron. Remarkably, our ELM neuron requires only 8K trainable parameters to match the aforementioned input-output relationship accurately. We find that an accurate model necessitates multiple memory-like hidden states and intricate nonlinear synaptic integration. To assess the computational ramifications of this design, we evaluate the ELM neuron on various tasks with demanding temporal structures, including a sequential version of the CIFAR-10 classification task, the challenging Pathfinder-X task, and a new dataset based on the Spiking Heidelberg Digits dataset. Our ELM neuron outperforms most transformer-based models on the Pathfinder-X task with 77% accuracy, demonstrates competitive performance on Sequential CIFAR-10, and superior performance compared to classic LSTM models on the variant of the Spiking Heidelberg Digits dataset. These findings indicate a potential for biologically motivated, computationally efficient neuronal models to enhance performance in challenging machine learning tasks.

翻译：传统的规模化神经科学模型与机器学习采用简化的单神经元模型，依赖集群活动与恰当调整的连接完成复杂计算。然而，每个生物皮层神经元本质上都是精密的计算设备——近期研究表明，需使用包含数百万参数的深层人工神经网络才能复现皮层锥体神经元详细生物物理模型的输入输出关系。我们质疑这种参数量的必要性，提出具有表达能力的漏记忆(ELM)神经元——一种受生物学启发、兼具计算表达能力与高效性的皮层神经元模型。值得注意的是，我们的ELM神经元仅需8K可训练参数便能精确匹配上述输入输出关系。研究发现，精确建模需要多个类记忆隐状态与复杂的非线性突触整合。为评估这一设计的计算影响，我们在多种具有挑战性时序结构的任务中测试ELM神经元，包括顺序版CIFAR-10分类任务、高难度Pathfinder-X任务以及基于脉冲海德堡数字数据集的新数据集。在Pathfinder-X任务上，ELM神经元以77%的准确率超越多数基于Transformer的模型；在顺序CIFAR-10上展现竞争性表现；在脉冲海德堡数字数据集的变体上优于经典LSTM模型。这些结果表明，受生物学启发的计算高效神经元模型有潜力提升机器学习挑战性任务的性能。