We propose a novel class of neural network-like parametrized functions, i.e., general transformation neural networks (GTNNs), for high-dimensional approximation. Conventional deep neural networks sometimes perform less accurately in approximation problems under gradient descent training, especially when the target function is oscillatory. To improve accuracy, we generalize the affine transformation of the abstract neuron to more general functions, which act as complex shape functions and have larger capacities. Specifically, we introduce two types of GTNNs: the cubic and quadratic transformation neural networks (CTNNs and QTNNs). We perform approximation error analysis for CTNNs and QTNNs, presenting their universal approximation properties for continuous functions and error bounds for smooth functions and Barron-type functions. Several numerical examples of regression problems and partial differential equations are presented, demonstrating that CTNNs/QTNNs have advantages in accuracy and robustness over conventional fully connected neural networks.

翻译：本文提出一类新颖的类神经网络参数化函数——广义变换神经网络（GTNN），用于高维逼近问题。传统深度神经网络在梯度下降训练下的逼近问题中有时表现欠佳，尤其在目标函数具有振荡特性时更为明显。为提高精度，我们将抽象神经元的仿射变换推广至更一般的函数，这些函数作为复杂形状函数具有更强的表达能力。具体而言，我们引入两类GTNN：三次变换神经网络与二次变换神经网络（CTNN与QTNN）。我们对CTNN与QTNN进行了逼近误差分析，证明了其对连续函数的通用逼近性质，并给出了光滑函数及Barron型函数的误差界。通过回归问题和偏微分方程的若干数值算例表明，CTNN/QTNN在精度与鲁棒性方面相较于传统全连接神经网络具有显著优势。