Recent work has examined language models from a linguistic perspective to better understand how they acquire language. Most existing benchmarks focus on judging grammatical acceptability, whereas the ability to interpret meanings conveyed by grammatical forms has received much less attention. We introduce the Linguistic Minimal-Pair Benchmark for Evaluating Constructional Understanding in Language Models (CxMP), a benchmark grounded in Construction Grammar that treats form-meaning pairings, or constructions, as fundamental linguistic units. CxMP evaluates whether models can interpret the semantic relations implied by constructions, using a controlled minimal-pair design across nine construction types, including the let-alone, caused motion, and ditransitive constructions. Our results show that while syntactic competence emerges early, constructional understanding develops more gradually and remains limited even in large language models (LLMs). CxMP thus reveals persistent gaps in how language models integrate form and meaning, providing a framework for studying constructional understanding and learning trajectories in language models.

翻译：近期研究从语言学视角考察语言模型，以更好地理解其语言习得机制。现有基准大多聚焦于语法可接受性判断，而对语法形式所传达意义的解释能力关注较少。本文提出用于评估语言模型构式理解能力的语言学最小对比基准（CxMP），该基准以构式语法为理论基础，将形式-意义配对（即构式）视为基本语言单位。CxMP通过涵盖九类构式（包括"let-alone"构式、致使移动构式与双及物构式）的受控最小对比设计，评估模型能否解读构式所隐含的语义关系。实验结果表明：虽然句法能力在早期即显现，但构式理解能力的发展更为渐进，即使在大语言模型（LLMs）中仍存在局限。CxMP由此揭示了语言模型在形式与意义整合方面存在的持续缺陷，为研究语言模型的构式理解与学习轨迹提供了系统框架。