Modern language models are capable of contextualizing words based on their surrounding context. However, this capability is often compromised due to semantic change that leads to words being used in new, unexpected contexts not encountered during pre-training. In this paper, we model \textit{semantic change} by studying the effect of unexpected contexts introduced by \textit{lexical replacements}. We propose a \textit{replacement schema} where a target word is substituted with lexical replacements of varying relatedness, thus simulating different kinds of semantic change. Furthermore, we leverage the replacement schema as a basis for a novel \textit{interpretable} model for semantic change. We are also the first to evaluate the use of LLaMa for semantic change detection.

翻译：现代语言模型能够根据词汇的上下文环境对其进行语境化处理。然而，由于语义变化导致词汇在预训练中未遇到的新且意想不到的上下文中使用，这一能力常受到损害。本文通过研究由词汇替换引入的意外上下文效应，对语义变化进行建模。我们提出了一种替换机制，将目标词替换为关联度不同的词汇替代项，从而模拟不同类型的语义变化。此外，我们利用该替换机制作为基础，提出了一种新颖的可解释的语义变化模型。我们也是首个评估LLaMa在语义变化检测中应用的研究。