Universality results for equivariant neural networks remain rare. Those that do exist typically hold only in restrictive settings: either they rely on regular or higher-order tensor representations, leading to impractically high-dimensional hidden spaces, or they target specialized architectures, often confined to the invariant setting. This work develops a more general account. For invariant networks, we establish a universality theorem under separation constraints, showing that the addition of a fully connected readout layer secures approximation within the class of separation-constrained continuous functions. For equivariant networks, where results are even scarcer, we demonstrate that standard separability notions are inadequate and introduce the sharper criterion of $\textit{entry-wise separability}$. We show that with sufficient depth or with the addition of appropriate readout layers, equivariant networks attain universality within the entry-wise separable regime. Together with prior results showing the failure of universality for shallow models, our findings identify depth and readout layers as a decisive mechanism for universality, additionally offering a unified perspective that subsumes and extends earlier specialized results.

翻译：等变神经网络的普适性结果仍然较为罕见。现有结果通常仅在限制性条件下成立：要么依赖于正则或高阶张量表示，导致隐藏空间维度过高而难以实际应用；要么针对特定架构，且常局限于不变性框架。本文提出一个更通用的解释。对于不变性网络，我们在分离约束条件下建立了普适性定理，表明全连接读出层的加入可确保在分离约束的连续函数类内实现逼近。对于结果更为稀缺的等变网络，我们证明了标准可分离性概念并不适用，并引入了更精确的$\textit{逐元素可分离性}$准则。研究表明，在具备足够深度或加入适当读出层的情况下，等变网络可在逐元素可分离函数类内达到普适性。结合先前关于浅层模型普适性失效的结果，我们的发现将深度与读出层确立为普适性的关键机制，同时提供了一个统一视角，涵盖并扩展了此前的专门性结果。