In recent years, software engineers have explored ways to assist quantum software programmers. Our goal in this paper is to continue this exploration and see if quantum software programmers deal with some problems plaguing classical programs. Specifically, we examine whether intermittently failing tests, i.e., flaky tests, affect quantum software development. To explore flakiness, we conduct a preliminary analysis of 14 quantum software repositories. Then, we identify flaky tests and categorize their causes and methods of fixing them. We find flaky tests in 12 out of 14 quantum software repositories. In these 12 repositories, the lower boundary of the percentage of issues related to flaky tests ranges between 0.26% and 1.85% per repository. We identify 46 distinct flaky test reports with 8 groups of causes and 7 common solutions. Further, we notice that quantum programmers are not using some of the recent flaky test countermeasures developed by software engineers. This work may interest practitioners, as it provides useful insight into the resolution of flaky tests in quantum programs. Researchers may also find the paper helpful as it offers quantitative data on flaky tests in quantum software and points to new research opportunities.
翻译:近年来,软件工程师已开始探索辅助量子软件程序员的方法。本文旨在延续这一探索,考察量子软件程序员是否面临经典程序中普遍存在的问题。具体而言,我们研究了间歇性失败的测试(即不稳定测试)是否会影响量子软件开发。为探究不稳定性现象,我们对14个量子软件仓库进行了初步分析,识别出其中的不稳定测试,并对其成因及修复方法进行了分类。结果显示,14个量子软件仓库中有12个存在不稳定测试。在这12个仓库中,不稳定测试相关问题占比的下限范围为每个仓库0.26%至1.85%。我们共识别出46份独立的不稳定测试报告,涵盖8类成因和7种常见解决方案。此外,我们发现量子程序员尚未采用软件工程师近期开发的部分不稳定测试应对措施。本研究为从业者提供了量子程序中不稳定测试解决路径的实用见解,同时为研究者提供了量子软件不稳定性的量化数据及新的研究契机。