Equivariant neural networks encode the intrinsic symmetry of data as an inductive bias, which has achieved impressive performance in wide domains. However, the understanding to their expressive power remains premature. Focusing on 2-layer ReLU networks, this paper investigates the impact of enforcing equivariance constraints on the expressive power. By examining the boundary hyperplanes and the channel vectors, we constructively demonstrate that enforcing equivariance constraints could undermine the expressive power. Naturally, this drawback can be compensated for by enlarging the model size -- we further prove upper bounds on the required enlargement for compensation. Surprisingly, we show that the enlarged neural architectures have reduced hypothesis space dimensionality, implying even better generalizability.

翻译：等变神经网络将数据的内在对称性作为归纳偏置进行编码，已在广泛领域取得显著性能。然而，对其表达能力的理解仍不成熟。本文聚焦于双层ReLU网络，研究施加等变性约束对表达能力的影响。通过分析边界超平面和通道向量，我们建设性地证明施加等变性约束可能削弱表达能力。这一缺陷自然可通过扩大模型规模进行补偿——我们进一步证明补偿所需扩大量级的上界。令人惊讶的是，我们表明扩大后的神经架构具有更低的假设空间维度，这意味着其泛化能力甚至更强。