Derivationally related words, such as "runner" and "running", exhibit semantic differences which also elicit different visual scenarios. In this paper, we ask whether Vision and Language (V\&L) models capture such distinctions at the morphological level, using a a new methodology and dataset. We compare the results from V\&L models to human judgements and find that models' predictions differ from those of human participants, in particular displaying a grammatical bias. We further investigate whether the human-model misalignment is related to model architecture. Our methodology, developed on one specific morphological contrast, can be further extended for testing models on capturing other nuanced language features.

翻译：派生相关词汇（如“runner”与“running”）存在语义差异，并诱发不同的视觉场景。本文通过新方法论与数据集，探究视觉-语言模型能否在形态层面捕捉此类区别。我们将模型结果与人类判断进行对比，发现模型预测与人类标注存在显著差异，尤其在语法偏误方面。进一步研究表明，人机偏差与模型架构相关。基于特定形态对比开发的方法可拓展至测试模型对其他语言微妙特征的捕捉能力。