Classical results in neural network approximation theory show how arbitrary continuous functions can be approximated by networks with a single hidden layer, under mild assumptions on the activation function. However, the classical theory does not give a constructive means to generate the network parameters that achieve a desired accuracy. Recent results have demonstrated that for specialized activation functions, such as ReLUs and some classes of analytic functions, high accuracy can be achieved via linear combinations of randomly initialized activations. These recent works utilize specialized integral representations of target functions that depend on the specific activation functions used. This paper defines mollified integral representations, which provide a means to form integral representations of target functions using activations for which no direct integral representation is currently known. The new construction enables approximation guarantees for randomly initialized networks for a variety of widely used activation functions.

翻译：经典的神经网络逼近理论表明，在激活函数的温和假设下，任意连续函数可以通过具有单隐层的网络进行逼近。然而，经典理论并未提供实现所需精度的网络参数生成的构造性方法。近期研究证明，对于特定的激活函数（如ReLU及某些解析函数类），通过随机初始化激活函数的线性组合可实现高精度逼近。这些工作利用了依赖于特定激活函数的目标函数的特殊积分表示。本文定义了磨光化积分表示，使得对于尚无直接积分表示的激活函数，能够构建目标函数的积分表示。新构造方法为多种常用激活函数的随机初始化网络提供了逼近性保证。