Citation analysis is widely used in research evaluation to assess the impact of scientific papers. These analyses rest on the assumption that citation decisions by authors are accurate, representing the flow of knowledge from cited to citing papers. However, in practice, researchers often cite for reasons that are not related to the fact that there has been (intellectual) input from previous papers. Citations made for rhetorical reasons or without reading the cited work compromise the value of citations as instrument for research evaluation. Past research on threats to the accuracy of citations has mainly focused on citation bias as the primary concern. In this paper, we argue that citation noise - the undesirable variance in citation decisions - represents an equally critical but underexplored challenge in citation analysis. We define and differentiate two types of citation noise: citation level noise and citation pattern noise. Each type of noise is described in terms of how it arises and the specific ways it can undermine the validity of citation-based research assessments. By conceptually differing citation noise from citation accuracy and citation bias, we propose a framework for the foundation of citation analysis. We discuss strategies and interventions to minimize citation noise, aiming to improve the reliability and validity of citation analysis in research evaluation. We recommend that the current professional reform movement in research evaluation such as the Coalition for Advancing Research Assessment (CoARA) pick up these strategies and interventions as an additional building block for careful, responsible use of bibliometric indicators in research evaluation.
翻译:引文分析被广泛用于研究评估中,以衡量科学论文的影响力。这类分析基于一个假设:作者的引用决策是准确的,能够体现从被引文献到引用文献的知识流动。然而在实践中,研究者常常出于与被引文献(智力)贡献无关的原因进行引用。出于修辞目的或未阅读原文的引用行为,会损害引文作为研究评估工具的价值。以往关于引文准确性威胁的研究主要聚焦于引文偏误。本文认为,引文噪声——即引用决策中非理想的变异——是引文分析中一个同等重要但尚未被充分探讨的挑战。我们定义并区分了两种引文噪声:引文水平噪声与引文模式噪声。针对每种噪声,我们阐述了其产生机制及其对基于引文的研究评估效度的具体影响方式。通过从概念上区分引文噪声、引文准确性与引文偏误,我们提出了一个构建引文分析基础的框架。我们探讨了减少引文噪声的策略与干预措施,旨在提升引文分析在研究评估中的信度与效度。我们建议当前研究评估领域的专业改革运动(如“推进研究评估联盟”CoARA)采纳这些策略与措施,将其作为在研究评估中审慎、负责任地使用文献计量指标的补充基础。