Drawing inspiration from our human brain that designs different neurons for different tasks, recent advances in deep learning have explored modifying a network's neurons to develop so-called task-driven neurons. Prototyping task-driven neurons (referred to as NeuronSeek) employs symbolic regression (SR) to discover the optimal neuron formulation and construct a network from these optimized neurons. Along this direction, this work replaces symbolic regression with tensor decomposition (TD) to discover optimal neuronal formulations, offering enhanced stability and faster convergence. Furthermore, we establish theoretical guarantees that modifying the aggregation functions with common activation functions can empower a network with a fixed number of parameters to approximate any continuous function with an arbitrarily small error, providing a rigorous mathematical foundation for the NeuronSeek framework. Extensive empirical evaluations demonstrate that our NeuronSeek-TD framework not only achieves superior stability, but also is competitive relative to the state-of-the-art models across diverse benchmarks. The code is available at https://github.com/HanyuPei22/NeuronSeek.

翻译：受人类大脑为不同任务设计不同神经元的启发，深度学习的最新进展探索了修改网络神经元以开发所谓的任务驱动神经元。原型化任务驱动神经元（称为NeuronSeek）采用符号回归（SR）来发现最优神经元形式，并基于这些优化神经元构建网络。沿此方向，本研究用张量分解（TD）替代符号回归来发现最优神经元形式，从而提供更强的稳定性和更快的收敛速度。此外，我们建立了理论保证：通过常见激活函数修改聚合函数，能够使具有固定参数数量的网络以任意小的误差逼近任意连续函数，这为NeuronSeek框架提供了严格的数学基础。大量的实证评估表明，我们的NeuronSeek-TD框架不仅实现了卓越的稳定性，而且在多样化的基准测试中与最先进的模型相比具有竞争力。代码可在 https://github.com/HanyuPei22/NeuronSeek 获取。