Oppenheim and Lim (1981) showed that natural images stay recognizable when reconstructed from their Fourier phase alone, while the magnitude carries little of their identity. We ask whether trained image classifiers reproduce this asymmetry inside their hidden layers, and we test it causally: given two images, we transplant the phase of one onto the magnitude of the other at a chosen layer and record which image the prediction follows. In PRISM2D, GFNet, and ViT-B/16 the prediction follows the phase or sign donor, and deleting all image-specific magnitude barely moves accuracy, so identity rides on phase while image-specific magnitude is largely dispensable to the readout. ResNet-50 at first seems to break the pattern, because transplanting sign after its ReLUs does nothing; a fair intervention before the ReLU reveals a strong latent sign code in the late blocks, and a DC-only control shows the readout consumes a channel-wise spatial average. Controls rule out the trivial case in which magnitude simply stops depending on the image. The architectures therefore share a phase/sign identity code but expose it in different bases, set by rectification and readout geometry, which gives a mechanistic account of the texture--shape gap between CNNs and attention models.
翻译:Oppenheim与Lim(1981)的研究表明,自然图像在仅保留傅里叶相位重建时仍可识别,而幅度几乎不承载图像特征。本研究探究训练后的图像分类器是否在其隐藏层中复现这种非对称性,并通过因果实验验证:给定两幅图像,在特定网络层中将一幅图像的相位移植到另一幅图像的幅度上,记录预测结果跟随哪幅图像。在PRISM2D、GFNet和ViT-B/16中,预测结果跟随相位或符号提供者,且删除所有图像特定幅度后分类准确率几无下降,表明特征识别依赖于相位,而图像特定幅度对读出层而言基本可舍弃。ResNet-50初看打破该模式——在ReLU后移植符号无效;但ReLU前的有效干预揭示深层块存在强潜在符号编码,而仅保留直流量(DC)的对照实验显示读出层消耗的是通道维度的空间平均值。对照验证排除了幅度完全失去图像依赖性的平凡情况。因此,不同架构共享相位/符号身份编码机制,但以不同基底呈现,其差异由整流函数与读出几何结构决定,这为CNN与注意力模型之间的纹理-形状差距提供了机理解释。