With the recent addition of Retrieval-Augmented Generation (RAG), the scope and importance of Information Retrieval (IR) has expanded. As a result, the importance of a deeper understanding of IR models also increases. However, interpretability in IR remains under-explored, especially when it comes to the models' inner mechanisms. In this paper, we explore the possibility of adapting Integrated Gradient-based methods in an IR context to identify the role of individual neurons within the model. In particular, we provide new insights into the role of what we call "relevance" neurons, as well as how they deal with unseen data. Finally, we carry out an in-depth pruning study to validate our findings.

翻译：随着检索增强生成（RAG）技术的引入，信息检索（IR）的范围与重要性已显著扩展。因此，深入理解信息检索模型的重要性也日益凸显。然而，信息检索领域的可解释性研究仍显不足，尤其是在模型内部机制方面。本文探讨了在信息检索背景下应用基于积分梯度的方法，以识别模型中单个神经元的作用。特别地，我们针对所谓“相关性”神经元的功能及其处理未见数据的方式提出了新的见解。最后，我们通过深入的剪枝研究验证了这些发现。