Many studies were recently done for investigating the properties of contextual language models but surprisingly, only a few of them consider the properties of these models in terms of semantic similarity. In this article, we first focus on these properties for English by exploiting the distributional principle of substitution as a probing mechanism in the controlled context of SemCor and WordNet paradigmatic relations. Then, we propose to adapt the same method to a more open setting for characterizing the differences between static and contextual language models.

翻译：最近为调查背景语言模型的特性进行了许多研究,但令人惊讶的是,只有少数研究从语义相似性的角度考虑这些模型的特性。在本条中,我们首先通过利用分配替代原则作为SemCor和WordNet范式关系控制背景下的探测机制,重点研究英语的这些特性。然后,我们提议对同一方法进行调整,以更开放地描述静态语言模型和背景语言模型之间的差异。