The set of functions parameterized by a linear fully-connected neural network is a determinantal variety. We investigate the subvariety of functions that are equivariant or invariant under the action of a permutation group. Examples of such group actions are translations or $90^\circ$ rotations on images. We describe such equivariant or invariant subvarieties as direct products of determinantal varieties, from which we deduce their dimension, degree, Euclidean distance degree, and their singularities. We fully characterize invariance for arbitrary permutation groups, and equivariance for cyclic groups. We draw conclusions for the parameterization and the design of equivariant and invariant linear networks in terms of sparsity and weight-sharing properties. We prove that all invariant linear functions can be parameterized by a single linear autoencoder with a weight-sharing property imposed by the cycle decomposition of the considered permutation. The space of rank-bounded equivariant functions has several irreducible components, so it can {\em not} be parameterized by a single network -- but each irreducible component can. Finally, we show that minimizing the squared-error loss on our invariant or equivariant networks reduces to minimizing the Euclidean distance from determinantal varieties via the Eckart--Young theorem.

翻译：由线性全连接神经网络参数化的函数集合构成行列式簇。本文研究了在置换群作用下具有等变性或不变性的函数子簇。此类群作用的实例包括图像上的平移或$90^\circ$旋转。我们将这类等变或不变子簇描述为行列式簇的直积，并据此推导了其维数、度数、欧几里得距离度数及其奇异性。我们完整刻画了任意置换群下的不变性以及循环群下的等变性，并基于稀疏性与权重共享特性，为等变与不变线性网络的参数化与设计提出了结论。我们证明：所有不变线性函数均可通过单个线性自编码器参数化，其中权重共享特性由所考虑置换的循环分解决定。秩有界等变函数空间具有若干不可约分量，因此无法由单一网络参数化——但每个不可约分量均可单独参数化。最后，我们证明：在等变或不变网络上最小化平方误差损失，可通过Eckart-Young定理转化为最小化与行列式簇之间的欧几里得距离。