We analyze the number of neurons that a ReLU neural network needs to approximate multivariate monomials. We establish an exponential lower bound for the complexity of any shallow network that approximates the product function $\vec{x} \to \prod_{i=1}^d x_i$ on a general compact domain. Furthermore, we prove that this lower bound does not hold for normalized O(1)-Lipschitz monomials (or equivalently, by restricting to the unit cube). These results suggest shallow ReLU networks suffer from the curse of dimensionality when expressing functions with a Lipschitz parameter scaling with the dimension of the input, and that the expressive power of neural networks lies in their depth rather than the overall complexity.

翻译：我们分析了ReLU神经网络逼近多元单项式所需的神经元数量。针对一般紧致域上的乘积函数$\vec{x} \to \prod_{i=1}^d x_i$，我们证明了任意浅层网络逼近该函数时存在指数级下界。进一步地，我们证明了该下界对于归一化O(1)-Lipschitz单项式（等价于限制在单位超立方体上）不成立。这些结果表明：当函数的Lipschitz参数随输入维度增长时，浅层ReLU网络会遭遇维度灾难；而神经网络的表达能力主要源自其深度而非整体复杂度。