Doubly-stochastic matrices (DSM) are increasingly utilized in structure-preserving deep architectures -- such as Optimal Transport layers and Sinkhorn-based attention -- to enforce numerical stability and probabilistic interpretability. In this work, we identify a critical spectral degradation phenomenon inherent to these constraints, termed the Homogeneity Trap. We demonstrate that the maximum-entropy bias, typical of Sinkhorn-based projections, drives the mixing operator towards the uniform barycenter, thereby suppressing the subdominant singular value σ_2 and filtering out high-frequency feature components. We derive a spectral bound linking σ_2 to the network's effective depth, showing that high-entropy constraints restrict feature transformation to a shallow effective receptive field. Furthermore, we formally demonstrate that Layer Normalization fails to mitigate this collapse in noise-dominated regimes; specifically, when spectral filtering degrades the Signal-to-Noise Ratio (SNR) below a critical threshold, geometric structure is irreversibly lost to noise-induced orthogonal collapse. Our findings highlight a fundamental trade-off between entropic stability and spectral expressivity in DSM-constrained networks.
翻译:双重随机矩阵(DSM)在结构保持的深度架构(如最优传输层和基于Sinkhorn的注意力机制)中的应用日益增多,以增强数值稳定性和概率可解释性。本文揭示了此类约束所固有的一个关键谱退化现象,称为“同质化陷阱”。我们证明,基于Sinkhorn投影中典型的极大熵偏置,会驱使混合算子趋向均匀重心,从而抑制次主导奇异值σ_2并滤除高频特征成分。我们推导出一个将σ_2与网络有效深度相关联的谱界,表明高熵约束会将特征变换限制在浅层有效感受野内。此外,我们严格证明了在噪声主导机制中,层归一化无法缓解此类塌缩;具体而言,当谱滤波使信噪比(SNR)降至临界阈值以下时,几何结构将因噪声诱导的正交塌缩而不可逆地丢失。我们的研究结果凸显了DSM约束网络中熵稳定性与谱表达能力之间的根本性权衡。