Prediction of image memorability has attracted interest in various fields. Consequently, the prediction accuracy of convolutional neural network (CNN) models has been approaching the empirical upper bound estimated based on human consistency. However, identifying which feature representations embedded in CNN models are responsible for the high memorability prediction accuracy remains an open question. To tackle this problem, we sought to identify memorability-related feature representations in CNN models using brain similarity. Specifically, memorability prediction accuracy and brain similarity were examined across 16,860 layers in 64 CNN models pretrained for object recognition. A clear tendency was observed in this comprehensive analysis that layers with high memorability prediction accuracy had higher brain similarity with the inferior temporal (IT) cortex, which is the highest stage in the ventral visual pathway. Furthermore, fine-tuning of the 64 CNN models for memorability prediction revealed that brain similarity with the IT cortex at the penultimate layer positively correlated with the memorability prediction accuracy of the models. This analysis also showed that the best fine-tuned model provided accuracy comparable to state-of-the-art CNN models developed for memorability prediction. Overall, the results of this study indicated that the CNN models' great success in predicting memorability relies on feature representation acquisition, similar to the IT cortex. This study advances our understanding of feature representations and their use in predicting image memorability.

翻译：图像记忆效用的预测在多个领域引起了广泛关注。目前，卷积神经网络模型的预测精度已接近基于人类一致性估计的经验上限。然而，识别CNN模型中哪些特征表示对高记忆效用预测精度负责仍是一个未解问题。为解决这一问题，我们尝试通过脑相似性识别CNN模型中与记忆效用相关的特征表示。具体而言，我们检测了64个预训练用于物体识别的CNN模型的16,860层中的记忆效用预测精度与脑相似性。这项全面分析显示了一个明显趋势：记忆效用预测精度高的层与腹侧视觉通路最高阶段——下颞叶皮层的脑相似性更高。此外，针对记忆效用预测对这64个CNN模型进行微调后发现，倒数第二层与IT皮层的脑相似性与模型记忆效用预测精度呈正相关。该分析还表明，最佳微调模型的精度与专为记忆效用预测开发的最新CNN模型相当。总体而言，本研究结果表明，CNN模型在记忆效用预测中的卓越表现依赖于与IT皮层相似的特征表示获取。这项研究推进了我们对特征表示及其在图像记忆效用预测中的应用的理解。