Do LMs infer the semantics of text from co-occurrence patterns in their training data? Merrill et al. (2022) argue that, in theory, probabilities predicted by an optimal LM encode semantic information about entailment relations, but it is unclear whether neural LMs trained on corpora learn entailment in this way because of strong idealizing assumptions made by Merrill et al. In this work, we investigate whether their theory can be used to decode entailment judgments from neural LMs. We find that a test similar to theirs can decode entailment relations between natural sentences, well above random chance, though not perfectly, across many datasets and LMs. This suggests LMs implicitly model aspects of semantics to predict semantic effects on sentence co-occurrence patterns. However, we find the test that predicts entailment in practice works in the opposite direction to the theoretical test. We thus revisit the assumptions underlying the original test, finding its derivation did not adequately account for redundancy in human-written text. We argue that correctly accounting for redundancy related to explanations might derive the observed flipped test and, more generally, improve linguistic theories of human speakers.

翻译：语言模型能否从训练数据中的共现模式推断文本语义？Merrill等人（2022）认为，理论上最优语言模型预测的概率编码了关于蕴涵关系的语义信息，但由于该研究存在较强的理想化假设，神经语言模型是否通过此方式从语料库中习得蕴涵尚不明确。本文探究了他们的理论能否用于从神经语言模型中解码蕴涵判断。我们发现，一种类似其理论的检测方法虽未达到完美效果，但能在多个数据集和语言模型中显著高于随机水平地解码自然语句间的蕴涵关系。这表明语言模型为预测句子共现模式中的语义效应，隐式地建模了语义的某些方面。然而，实际预测蕴涵的检测方向与理论检测方向相反。因此我们重新审视原始检测的假设基础，发现其推导过程未能充分解释人类文本中的冗余性。我们认为，若正确解释与解释性内容相关的冗余现象，不仅能推导出观测到的翻转检测结果，还可普遍改进人类说话者的语言学理论。