We study the universality of complex-valued neural networks with bounded widths and arbitrary depths. Under mild assumptions, we give a full description of those activation functions $\varrho:\mathbb{C}\to \mathbb{C}$ that have the property that their associated networks are universal, i.e., are capable of approximating continuous functions to arbitrary accuracy on compact domains. Precisely, we show that deep narrow complex-valued networks are universal if and only if their activation function is neither holomorphic, nor antiholomorphic, nor $\mathbb{R}$-affine. This is a much larger class of functions than in the dual setting of arbitrary width and fixed depth. Unlike in the real case, the sufficient width differs significantly depending on the considered activation function. We show that a width of $2n+2m+5$ is always sufficient and that in general a width of $\max\{2n,2m\}$ is necessary. We prove, however, that a width of $n+m+4$ suffices for a rich subclass of the admissible activation functions. Here, $n$ and $m$ denote the input and output dimensions of the considered networks.

翻译：我们研究了具有有界宽度和任意深度的复值神经网络的通用性。在温和假设下，我们完整刻画了那些使其关联网络具有通用性的激活函数$\varrho:\mathbb{C}\to \mathbb{C}$的性质，即能够在紧致域上以任意精度逼近连续函数。具体而言，我们证明深层窄复值网络是通用的当且仅当其激活函数既非全纯、也非反全纯、亦非$\mathbb{R}$-仿射。相较于宽度任意、深度固定的对偶设定，此类函数类别更为广泛。与实数情形不同，充分的网络宽度会因所选激活函数而有显著差异。我们证明宽度$2n+2m+5$总是充分的，且一般而言宽度$\max\{2n,2m\}$是必要的。然而我们证明，对于可容许激活函数的一个丰富子类而言，宽度$n+m+4$即足以保证通用性。此处$n$和$m$分别表示所考虑网络的输入与输出维度。