The approximation properties of infinitely wide shallow neural networks heavily depend on the choice of the activation function. To understand this influence, we study embeddings between Barron spaces with different activation functions. These embeddings are proven by providing push-forward maps on the measures $\mu$ used to represent functions $f$. An activation function of particular interest is the rectified power unit ($\operatorname{RePU}$) given by $\operatorname{RePU}_s(x)=\max(0,x)^s$. For many commonly used activation functions, the well-known Taylor remainder theorem can be used to construct a push-forward map, which allows us to prove the embedding of the associated Barron space into a Barron space with a $\operatorname{RePU}$ as activation function. Moreover, the Barron spaces associated with the $\operatorname{RePU}_s$ have a hierarchical structure similar to the Sobolev spaces $H^m$.

翻译：无限宽浅神经网络逼近特性高度依赖于激活函数的选择。为理解这种影响，我们研究了具有不同激活函数的Barron空间之间的嵌入关系。这些嵌入通过构造用于表示函数$f$的测度$\mu$上的前推映射得以证明。特别关注的激活函数是由$\operatorname{RePU}_s(x)=\max(0,x)^s$定义的整流幂单元。对于许多常用激活函数，可利用经典的泰勒余项定理构造前推映射，从而证明相关Barron空间可嵌入到以$\operatorname{RePU}$为激活函数的Barron空间中。此外，与$\operatorname{RePU}_s$关联的Barron空间具有类似Sobolev空间$H^m$的层次结构。