For software testing research, Defects4J stands out as the primary benchmark dataset, offering a controlled environment to study real bugs from prominent open-source systems. However, prior research indicates that Defects4J might include tests added post-bug report, embedding developer knowledge and affecting fault localization efficacy. In this paper, we examine Defects4J's fault-triggering tests, emphasizing the implications of developer knowledge of SBFL techniques. We study the timelines of changes made to these tests concerning bug report creation. Then, we study the effectiveness of SBFL techniques without developer knowledge in the tests. We found that 1) 55% of the fault-triggering tests were newly added to replicate the bug or to test for regression; 2) 22% of the fault-triggering tests were modified after the bug reports were created, containing developer knowledge of the bug; 3) developers often modify the tests to include new assertions or change the test code to reflect the changes in the source code; and 4) the performance of SBFL techniques degrades significantly (up to --415% for Mean First Rank) when evaluated on the bugs without developer knowledge. We provide a dataset of bugs without developer insights, aiding future SBFL evaluations in Defects4J and informing considerations for future bug benchmarks.

翻译：在软件测试研究中，Defects4J作为核心基准数据集，为研究来自知名开源系统的真实缺陷提供了受控环境。然而，已有研究表明Defects4J可能包含缺陷报告发布后添加的测试用例，这些测试嵌入了开发者的知识，从而影响了故障定位的有效性。本文聚焦Defects4J中的故障触发测试，重点分析开发者对SBFL技术认知所造成的影响。我们首先梳理这些测试的变更时间线与缺陷报告创建时间的关系，进而研究在测试中剔除开发者知识后SBFL技术的实际效能。研究发现：1）55%的故障触发测试是新增的，用于复现缺陷或回归验证；2）22%的故障触发测试在缺陷报告创建后被修改，其中包含开发者对缺陷的认知；3）开发者常通过添加新断言或调整测试代码来适配源代码变更；4）在无开发者知识注入的缺陷评估中，SBFL技术性能显著下降（平均首次排名指标降幅达-415%）。本研究提供了无开发者知识偏差的缺陷数据集，为后续基于Defects4J的SBFL评估提供支撑，并为未来缺陷基准构建提供参考依据。