Despite their impressive performance on computer vision benchmarks, Deep Neural Networks (DNNs) still fall short of adequately modeling human visual behavior, as measured by error consistency and shape bias. Recent work hypothesized that behavioral alignment can be drastically improved through \emph{generative} -- rather than \emph{discriminative} -- classifiers, with far-reaching implications for models of human vision. Here, we instead show that the increased alignment of generative models can be largely explained by a seemingly innocuous resizing operation in the generative model which effectively acts as a low-pass filter. In a series of controlled experiments, we show that removing high-frequency spatial information from discriminative models like CLIP drastically increases their behavioral alignment. Simply blurring images at test-time -- rather than training on blurred images -- achieves a new state-of-the-art score on the model-vs-human benchmark, halving the current alignment gap between DNNs and human observers. Furthermore, low-pass filters are likely optimal, which we demonstrate by directly optimizing filters for alignment. To contextualize the performance of optimal filters, we compute the frontier of all possible pareto-optimal solutions to the benchmark, which was formerly unknown. We explain our findings by observing that the frequency spectrum of optimal Gaussian filters roughly matches the spectrum of band-pass filters implemented by the human visual system. We show that the contrast sensitivity function, describing the inverse of the contrast threshold required for humans to detect a sinusoidal grating as a function of spatiotemporal frequency, is approximated well by Gaussian filters of the specific width that also maximizes error consistency.

翻译：尽管深度神经网络（DNN）在计算机视觉基准测试中表现出色，但在误差一致性和形状偏差等指标上，它们仍未能充分模拟人类的视觉行为。近期研究假设，通过使用**生成式**而非**判别式**分类器，可以大幅提升行为对齐度，这对人类视觉模型具有深远意义。然而，本文研究表明，生成模型对齐度的提升很大程度上可归因于其中一种看似无害的调整大小操作，该操作实际上起到了低通滤波器的作用。在一系列受控实验中，我们发现从CLIP等判别模型中移除高频空间信息能显著提高其行为对齐度。仅在测试时对图像进行模糊处理——而非在训练时使用模糊图像——便在模型与人类基准测试中取得了新的最优成绩，将DNN与人类观察者之间的当前对齐差距缩小了一半。此外，低通滤波器很可能是最优选择，我们通过直接优化滤波器以提升对齐度证明了这一点。为评估最优滤波器的性能，我们计算了该基准测试中所有可能的帕累托最优解的前沿，这一前沿此前未知。我们通过观察发现，最优高斯滤波器的频谱大致匹配了人类视觉系统实现的带通滤波器频谱，从而解释了上述结果。研究表明，描述人类检测正弦光栅所需对比度阈值倒数随时空频率变化的对比敏感度函数，可由特定宽度的高斯滤波器良好近似，该宽度同时也能最大化误差一致性。