Deep neural networks (DNNs) have been widely deployed across diverse domains such as computer vision and natural language processing. However, the impressive accomplishments of DNNs have been realized alongside extensive computational demands, thereby impeding their applicability on resource-constrained devices. To address this challenge, many researchers have been focusing on basic neuron structures, the fundamental building blocks of neural networks, to alleviate the computational and storage cost. In this work, an efficient quadratic neuron architecture distinguished by its enhanced utilization of second-order computational information is introduced. By virtue of their better expressivity, DNNs employing the proposed quadratic neurons can attain similar accuracy with fewer neurons and computational cost. Experimental results have demonstrated that the proposed quadratic neuron structure exhibits superior computational and storage efficiency across various tasks when compared with both linear and non-linear neurons in prior work.

翻译：深度神经网络（DNNs）已广泛应用于计算机视觉和自然语言处理等多个领域。然而，DNNs的显著成就伴随着巨大的计算需求，从而限制了其在资源受限设备上的适用性。为应对这一挑战，众多研究者聚焦于神经网络的基本构建单元——基本神经元结构，以降低计算与存储成本。本文提出了一种高效的二次神经元架构，其独特之处在于能更充分地利用二阶计算信息。凭借更强的表达能力，采用所提二次神经元的DNNs能够以更少的神经元和计算量达到相似精度。实验结果表明，与以往工作中的线性和非线性神经元相比，所提出的二次神经元结构在各类任务中均展现出更优的计算与存储效率。