In the past decade, many successful networks are on novel architectures, which almost exclusively use the same type of neurons. Recently, more and more deep learning studies have been inspired by the idea of NeuroAI and the neuronal diversity observed in human brains, leading to the proposal of novel artificial neuron designs. Designing well-performing neurons represents a new dimension relative to designing well-performing neural architectures. Biologically, the brain does not rely on a single type of neuron that universally functions in all aspects. Instead, in our brain, neurons are often task-based. In this study, we address the following question: since the human brain is a task-based neuron user, can the artificial network design go from the task-based architecture design to the task-based neuron design? Since methodologically there are no one-size-fits-all neurons, given the same structure, task-based neurons can enhance the feature representation ability relative to the existing universal neurons due to the intrinsic inductive bias for the task. Specifically, we propose a two-step framework for prototyping task-based neurons. As the initial step, we evaluate the proposed framework using polynomials as base functions. Empirically, systematic experimental results on synthetic data, classic benchmarks, and real-world applications show that the proposed task-based neuron design is not only feasible but also delivers competitive performance over other state-of-the-art models.

翻译：近十年来，许多成功的网络依赖于新颖架构，这些架构几乎无一例外地使用相同类型的神经元。近期，越来越多的深度学习研究受到神经人工智能（NeuroAI）理念以及人脑中神经元多样性的启发，从而提出了多种新型人工神经元设计。设计性能优异的神经元代表了相对于设计优异神经架构的一个新维度。从生物学角度看，大脑并非依赖一种在所有方面通用功能的单一类型神经元；相反，在我们的大脑中，神经元往往是基于任务的。在本研究中，我们探讨以下问题：既然人脑是一种基于任务的神经元使用者，那么人工网络设计能否从基于任务的架构设计走向基于任务的神经元设计？由于方法论上不存在“一刀切”的神经元，在相同结构下，基于任务的神经元因其对任务固有的归纳偏置，相比现有的通用神经元能够增强特征表示能力。具体而言，我们提出了一个用于原型化基于任务的神经元的两步框架。作为初始步骤，我们使用多项式作为基函数对提出的框架进行评估。实验上，在合成数据、经典基准测试以及实际应用中的系统性实验结果表明，所提出的基于任务的神经元设计不仅可行，而且在性能上可与其他最先进模型相媲美。