Understanding where transformer language models encode psychologically meaningful aspects of meaning is essential for both theory and practice. We conduct a systematic layer-wise probing study of 58 psycholinguistic features across 10 transformer models, spanning encoder-only and decoder-only architectures, and compare three embedding extraction methods. We find that apparent localization of meaning is strongly method-dependent: contextualized embeddings yield higher feature-specific selectivity and different layer-wise profiles than isolated embeddings. Across models and methods, final-layer representations are rarely optimal for recovering psycholinguistic information with linear probes. Despite these differences, models exhibit a shared depth ordering of meaning dimensions, with lexical properties peaking earlier and experiential and affective dimensions peaking later. Together, these results show that where meaning "lives" in transformer models reflects an interaction between methodological choices and architectural constraints.

翻译：理解Transformer语言模型在何处编码具有心理学意义的意义层面，对于理论研究和实际应用都至关重要。本研究对10个Transformer模型（涵盖仅编码器和仅解码器架构）的58个心理语言学特征进行了系统的分层探测分析，并比较了三种嵌入提取方法。研究发现：意义的表观定位具有强烈的方法依赖性——与孤立嵌入相比，上下文嵌入展现出更高的特征选择特异性及不同的分层分布模式。在所有模型与方法中，最终层表征很少能通过线性探测器最优地还原心理语言学信息。尽管存在这些差异，各模型在意义维度的深度排序上表现出共性：词汇属性在较浅层达到峰值，而经验性与情感性维度则在较深层达到峰值。综合而言，这些结果表明Transformer模型中意义“栖居”的位置反映了方法论选择与架构约束之间的交互作用。