The celebrated universal approximation theorems for neural networks roughly state that any reasonable function can be arbitrarily well-approximated by a network whose parameters are appropriately chosen real numbers. This paper examines the approximation capabilities of one-bit neural networks -- those whose nonzero parameters are $\pm a$ for some fixed $a\not=0$. One of our main theorems shows that for any $f\in C^s([0,1]^d)$ with $\|f\|_\infty<1$ and error $\varepsilon$, there is a $f_{NN}$ such that $|f(\boldsymbol{x})-f_{NN}(\boldsymbol{x})|\leq \varepsilon$ for all $\boldsymbol{x}$ away from the boundary of $[0,1]^d$, and $f_{NN}$ is either implementable by a $\{\pm 1\}$ quadratic network with $O(\varepsilon^{-2d/s})$ parameters or a $\{\pm \frac 1 2 \}$ ReLU network with $O(\varepsilon^{-2d/s}\log (1/\varepsilon))$ parameters, as $\varepsilon\to0$. We establish new approximation results for iterated multivariate Bernstein operators, error estimates for noise-shaping quantization on the Bernstein basis, and novel implementation of the Bernstein polynomials by one-bit quadratic and ReLU neural networks.

翻译：著名的神经网络通用逼近定理大致表明，任何合理函数均可通过参数适当选取为实数的网络实现任意精度的逼近。本文研究单比特神经网络（即非零参数固定为±a，其中a≠0）的逼近能力。我们的主要定理之一表明：对任意满足‖f‖∞<1的f∈C^s([0,1]^d)及误差ε，存在神经网络fNN使得对所有远离[0,1]^d边界的x满足|f(x)-fNN(x)|≤ε。当ε→0时，该网络可通过参数量为O(ε^{-2d/s})的{±1}二次网络或参数量为O(ε^{-2d/s} log(1/ε))的{±1/2} ReLU网络实现。本文建立了迭代多元伯恩斯坦算子的新逼近结果、伯恩斯坦基上噪声整形量化的误差估计，以及通过单比特二次神经网络和ReLU神经网络实现伯恩斯坦多项式的新方法。