Word co-occurrence patterns in language corpora contain a surprising amount of conceptual knowledge. Large language models (LLMs), trained to predict words in context, leverage these patterns to achieve impressive performance on diverse semantic tasks requiring world knowledge. An important but understudied question about LLMs' semantic abilities is whether they acquire generalized knowledge of common events. Here, we test whether five pre-trained LLMs (from 2018's BERT to 2023's MPT) assign higher likelihood to plausible descriptions of agent-patient interactions than to minimally different implausible versions of the same event. Using three curated sets of minimal sentence pairs (total n=1,215), we found that pre-trained LLMs possess substantial event knowledge, outperforming other distributional language models. In particular, they almost always assign higher likelihood to possible vs. impossible events (The teacher bought the laptop vs. The laptop bought the teacher). However, LLMs show less consistent preferences for likely vs. unlikely events (The nanny tutored the boy vs. The boy tutored the nanny). In follow-up analyses, we show that (i) LLM scores are driven by both plausibility and surface-level sentence features, (ii) LLM scores generalize well across syntactic variants (active vs. passive constructions) but less well across semantic variants (synonymous sentences), (iii) some LLM errors mirror human judgment ambiguity, and (iv) sentence plausibility serves as an organizing dimension in internal LLM representations. Overall, our results show that important aspects of event knowledge naturally emerge from distributional linguistic patterns, but also highlight a gap between representations of possible/impossible and likely/unlikely events.

翻译：语言语料库中的词共现模式包含大量概念性知识。大型语言模型（LLMs）经过训练以预测上下文中的词汇，利用这些模式在需要世界知识的多样化语义任务上取得了令人瞩目的表现。关于LLMs语义能力的一个重要但尚未充分研究的问题是，它们是否习得了关于常见事件的概括性知识。在此，我们测试了五个预训练LLMs（从2018年的BERT到2023年的MPT）是否会为合理的施事-受事交互描述赋予比该事件最小差异的不可信版本更高的概率。使用三组精心构建的最小句子对（总计n=1,215），我们发现预训练LLMs具备显著的事件知识，其表现优于其他分布式语言模型。特别是，它们几乎总是为可能事件与不可能事件赋予更高的概率（如“老师买了笔记本电脑” vs. “笔记本电脑买了老师”）。然而，对于可能发生事件与不太可能发生事件（如“保姆辅导了男孩” vs. “男孩辅导了保姆”），LLMs表现出较不稳定的一致性偏好。在后续分析中，我们表明：（i）LLM评分同时受可信度和句子表层特征驱动；（ii）LLM评分在句法变体（主动与被动结构）上泛化良好，但在语义变体（同义句）上泛化较差；（iii）部分LLM错误反映了人类判断的模糊性；（iv）句子可信度作为LLM内部表征中的一个组织维度。总体而言，我们的结果表明，事件知识的重要方面自然地从分布语言模式中涌现，但也凸显了可能/不可能事件与可能/不太可能事件表征之间的差距。