Biologically, the brain does not rely on a single type of neuron that universally functions in all aspects. Instead, it acts as a sophisticated designer of task-based neurons. In this study, we address the following question: since the human brain is a task-based neuron user, can the artificial network design go from the task-based architecture design to the task-based neuron design? Since methodologically there are no one-size-fits-all neurons, given the same structure, task-based neurons can enhance the feature representation ability relative to the existing universal neurons due to the intrinsic inductive bias for the task. Specifically, we propose a two-step framework for prototyping task-based neurons. First, symbolic regression is used to identify optimal formulas that fit input data by utilizing base functions such as logarithmic, trigonometric, and exponential functions. We introduce vectorized symbolic regression that stacks all variables in a vector and regularizes each input variable to perform the same computation, which can expedite the regression speed, facilitate parallel computation, and avoid overfitting. Second, we parameterize the acquired elementary formula to make parameters learnable, which serves as the aggregation function of the neuron. The activation functions such as ReLU and the sigmoidal functions remain the same because they have proven to be good. Empirically, experimental results on synthetic data, classic benchmarks, and real-world applications show that the proposed task-based neuron design is not only feasible but also delivers competitive performance over other state-of-the-art models.

翻译：生物学上，大脑并非依赖单一类型的神经元来普遍适用于所有方面。相反，它扮演着任务型神经元的精细设计者角色。在本研究中，我们探讨以下问题：既然人脑是任务型神经元的使用者，人工网络设计能否从任务型架构设计转向任务型神经元设计？由于方法论上不存在放之四海而皆准的神经元，在相同结构下，任务型神经元凭借其对任务固有的归纳偏置，相比现有通用神经元可增强特征表示能力。具体而言，我们提出一个用于原型设计任务型神经元的两步框架。首先，利用符号回归通过对数、三角函数和指数函数等基函数来识别拟合输入数据的最优公式。我们引入向量化符号回归，将所有变量堆叠为向量，并正则化每个输入变量以执行相同运算，从而加快回归速度、促进并行计算并避免过拟合。其次，我们将获取的基础公式参数化，使其参数可学习，从而作为神经元的聚合函数。ReLU和sigmoid等激活函数保持不变，因为它们已被证明确实有效。实验方面，在合成数据、经典基准测试和实际应用上的结果表明，所提出的任务型神经元设计不仅可行，而且相比其他最先进模型展现出具有竞争力的性能。