Traceability information between requirements and source code greatly benefits the maintenance of a software system. Since manually establishing trace links is cumbersome and error-prone, previous research explored automated traceability link recovery (TLR) approaches to support this task. However, quality defects in requirements impact subsequent activities such as TLR, yet evidence about this remains scarce. Our objective is to contribute empirical evidence on this impact. At the same time, we aim to understand how the performance of TLR approaches varies given these quality defects. To this end, we annotated 28 types of quality defect in 189 use case descriptions from two datasets. Then, we executed five distinct TLR approaches on the dataset and measured their performance in recovering trace links. Finally, we performed statistical tests to quantify the defects' effect strength on this performance. Our results show that some quality defects harm TLR performance, e.g., sentences that do not start with noun phrases, while others actually benefit performance, e.g., use cases that include implementation details. Moreover, different types of approaches respond differently to these defects. As a consequence, the performance-optimizing choice of a TLR approach depends on the quality of the dataset.
翻译:需求与源代码之间的可追溯性信息对软件系统的维护大有裨益。由于手动建立可追溯链接既繁琐又易出错,先前的研究探索了自动化可追溯性链接恢复(TLR)方法来支持这一任务。然而,需求中的质量缺陷会影响后续活动(如TLR),但相关证据仍然匮乏。我们的目标是为此影响提供经验性证据。同时,我们旨在理解给定这些质量缺陷后,TLR方法的表现如何变化。为此,我们在来自两个数据集的189个用例描述中标注了28种质量缺陷。随后,我们在数据集上执行了五种不同的TLR方法,并测量了它们在恢复可追溯链接方面的性能。最后,我们进行统计检验以量化缺陷对该性能的影响强度。结果表明,某些质量缺陷会损害TLR性能(例如,不以名词短语开头的句子),而其他缺陷实际上有利于性能(例如,包含实现细节的用例)。此外,不同类型的方法对这些缺陷的反应不同。因此,TLR方法的性能优化选择取决于数据集的质量。