For the past 30 years or so, machine learning has stimulated a great deal of research in the study of approximation capabilities (expressive power) of a multitude of processes, such as approximation by shallow or deep neural networks, radial basis function networks, and a variety of kernel based methods. Motivated by applications such as invariant learning, transfer learning, and synthetic aperture radar imaging, we initiate in this paper a general approach to study the approximation capabilities of kernel based networks using non-symmetric kernels. While singular value decomposition is a natural instinct to study such kernels, we consider a more general approach to include the use of a family of kernels, such as generalized translation networks (which include neural networks and translation invariant kernels as special cases) and rotated zonal function kernels. Naturally, unlike traditional kernel based approximation, we cannot require the kernels to be positive definite. In particular, we obtain estimates on the accuracy of uniform approximation of functions in a ($L^2$)-Sobolev class by ReLU$^r$ networks when $r$ is not necessarily an integer. Our general results apply to the approximation of functions with small smoothness compared to the dimension of the input space.

翻译：近三十年来，机器学习极大地推动了各类过程逼近能力（表达能力）的研究，包括浅层或深度神经网络逼近、径向基函数网络逼近以及多种基于核的方法。受不变学习、迁移学习和合成孔径雷达成像等应用的驱动，本文首次提出了一种通用框架，用于研究基于非对称核的核网络逼近能力。尽管奇异值分解是研究此类核的自然直觉方法，但我们提出了更通用的方法，涵盖了一族核的使用，例如广义平移网络（包含神经网络和平移不变核作为特例）以及旋转带状函数核。自然地，与传统基于核的逼近不同，我们无需要求核具有正定性。特别地，我们获得了ReLU^r网络对$L^2$阶Sobolev类函数一致逼近精度的估计，其中$r$不一定是整数。我们的通用结果适用于输入空间维度较高时具有低光滑度函数的逼近问题。