Recently, Locate-Then-Edit paradigm has emerged as one of the main approaches in changing factual knowledge stored in the Language models. However, there is a lack of research on whether present locating methods can pinpoint the exact parameters embedding the desired knowledge. Moreover, although many researchers have questioned the validity of locality hypothesis of factual knowledge, no method is provided to test the a hypothesis for more in-depth discussion and research. Therefore, we introduce KLoB, a benchmark examining three essential properties that a reliable knowledge locating method should satisfy. KLoB can serve as a benchmark for evaluating existing locating methods in language models, and can contributes a method to reassessing the validity of locality hypothesis of factual knowledge. Our is publicly available at \url{https://github.com/juyiming/KLoB}.

翻译：近年来，“定位-编辑”范式已成为修改语言模型中存储的事实知识的主要方法之一。然而，当前定位方法能否精准定位嵌入所需知识的具体参数尚缺乏研究。此外，尽管许多研究者对事实知识的可定位性假设的有效性提出了质疑，但目前尚无方法可用于检验该假设以进行更深入的探讨与研究。为此，我们提出了KLoB——一个用于检验可靠的知识定位方法应满足的三个核心属性的基准。KLoB可作为评估语言模型中现有定位方法的基准，并为重新评估事实知识可定位性假设的有效性提供了一种方法。我们的代码开源在 \url{https://github.com/juyiming/KLoB}。