In order to classify linearly non-separable data, neurons are typically organized into multi-layer neural networks that are equipped with at least one hidden layer. Inspired by some recent discoveries in neuroscience, we propose a new model of artificial neuron along with a novel activation function enabling the learning of nonlinear decision boundaries using a single neuron. We show that a standard neuron followed by our novel apical dendrite activation (ADA) can learn the XOR logical function with 100% accuracy. Furthermore, we conduct experiments on six benchmark data sets from computer vision, signal processing and natural language processing, i.e. MOROCO, UTKFace, CREMA-D, Fashion-MNIST, Tiny ImageNet and ImageNet, showing that the ADA and the leaky ADA functions provide superior results to Rectified Linear Units (ReLU), leaky ReLU, RBF and Swish, for various neural network architectures, e.g. one-hidden-layer or two-hidden-layer multi-layer perceptrons (MLPs) and convolutional neural networks (CNNs) such as LeNet, VGG, ResNet and Character-level CNN. We obtain further performance improvements when we change the standard model of the neuron with our pyramidal neuron with apical dendrite activations (PyNADA). Our code is available at: https://github.com/raduionescu/pynada.

翻译：为了对线性不可分数据进行分类，神经元通常被组织成至少包含一个隐藏层的多层神经网络。受神经科学近期发现的启发，我们提出了一种新的人工神经元模型及其新型激活函数，使得单个神经元能够学习非线性决策边界。我们证明，标准神经元后接我们提出的顶树突激活（ADA）能以100%的准确率学习XOR逻辑函数。此外，我们在计算机视觉、信号处理和自然语言处理领域的六个基准数据集（即MOROCO、UTKFace、CREMA-D、Fashion-MNIST、Tiny ImageNet和ImageNet）上进行了实验，结果表明，对于各种神经网络架构（如单隐藏层或多隐藏层多层感知器（MLP）以及卷积神经网络（CNN），包括LeNet、VGG、ResNet和字符级CNN），ADA和leaky ADA函数在性能上优于整流线性单元（ReLU）、leaky ReLU、RBF和Swish。当我们采用具有顶树突激活的锥体神经元（PyNADA）替代标准神经元模型时，我们获得了进一步的性能提升。我们的代码可从以下地址获取：https://github.com/raduionescu/pynada。