In quantum software engineering (QSE), quantum software testing (QST) has attracted increasing attention as quantum software systems grow in scale and complexity. Since QST evaluates quantum programs through execution under designed test inputs, empirical studies are widely used to assess the effectiveness of testing approaches. However, the design and reporting of empirical studies in QST remain highly diverse, and a shared methodological understanding has yet to emerge, making it difficult to interpret results and compare findings across studies. This paper presents a methodological analysis of empirical studies in QST through a systematic examination of 59 primary studies identified from a literature pool of size 384. We organize our analysis around ten research questions that cover key methodological dimensions of QST empirical studies, including objects under test, baseline comparison, testing setup, experimental configuration, and tool and artifact support. Through cross-study analysis along these dimensions, we characterize current empirical practices in QST, identify recurring limitations and inconsistencies, and highlight open methodological challenges. Based on our findings, we derive insights and recommendations to inform the design, execution, and reporting of future empirical studies in QST.
翻译:在量子软件工程领域,随着量子软件系统规模和复杂性的增长,量子软件测试日益受到关注。由于量子软件测试通过在设计测试输入下执行量子程序进行评估,实证研究被广泛用于评估测试方法的有效性。然而,量子软件测试中实证研究的设计与报告方式仍存在高度多样性,尚未形成统一的方法学共识,导致研究结果难以解释且跨研究比较困难。本文通过对384篇文献库中筛选出的59项核心研究进行系统性审查,开展了量子软件测试实证研究的方法学分析。我们围绕十个研究问题组织分析框架,这些问题涵盖了量子软件测试实证研究的关键方法学维度,包括被测对象、基线比较、测试设置、实验配置以及工具与制品支持。通过对这些维度的跨研究分析,我们刻画了当前量子软件测试实证研究的实践特征,识别了普遍存在的局限性与不一致现象,并指出了开放的方法学挑战。基于研究发现,我们提出了一系列见解与建议,以指导未来量子软件测试实证研究的设计、实施与报告工作。