By universal formulas we understand parameterized analytic expressions that have a fixed complexity, but nevertheless can approximate any continuous function on a compact set. There exist various examples of such formulas, including some in the form of neural networks. In this paper we analyze the essential structural elements of these highly expressive models. We introduce a hierarchy of expressiveness classes connecting the global approximability property to the weaker property of infinite VC dimension, and prove a series of classification results for several increasingly complex functional families. In particular, we introduce a general family of polynomially-exponentially-algebraic functions that, as we prove, is subject to polynomial constraints. As a consequence, we show that fixed-size neural networks with not more than one layer of neurons having transcendental activations (e.g., sine or standard sigmoid) cannot in general approximate functions on arbitrary finite sets. On the other hand, we give examples of functional families, including two-hidden-layer neural networks, that approximate functions on arbitrary finite sets, but fail to do that on the whole domain of definition.

翻译：我们所说的通用公式，是指具有固定复杂度但能在紧集上逼近任意连续函数的参数化解析表达式。存在多种此类公式的实例，包括一些以神经网络形式呈现的公式。本文分析了这些高表达能力模型的基本结构要素。我们引入了一个表达能力类的层级结构，将全局可逼近性与较弱的无限VC维性质联系起来，并针对几个复杂度递增的函数族证明了一系列分类结果。特别地，我们引入了一个广义的多项式-指数-代数函数族，并证明该函数族受多项式约束条件限制。由此得出结论：仅包含不多于一层具有超越激活函数（例如正弦函数或标准S型函数）神经元的固定规模神经网络，通常无法在任意有限集上逼近函数。另一方面，我们给出了包括双隐层神经网络在内的函数族实例，这些函数族能在任意有限集上逼近函数，但无法在整个定义域上实现逼近。