Despite its massive popularity as a programming language, especially in novel domains like data science programs, there is comparatively little research about fault localization that targets Python. Even though it is plausible that several findings about programming languages like C/C++ and Java -- the most common choices for fault localization research -- carry over to other languages, whether the dynamic nature of Python and how the language is used in practice affect the capabilities of classic fault localization approaches remain open questions to investigate. This paper is the first large-scale empirical study of fault localization on real-world Python programs and faults. Using Zou et al.'s recent large-scale empirical study of fault localization in Java as the basis of our study, we investigated the effectiveness (i.e., localization accuracy), efficiency (i.e., runtime performance), and other features (e.g., different entity granularities) of seven well-known fault-localization techniques in four families (spectrum-based, mutation-based, predicate switching, and stack-trace based) on 135 faults from 13 open-source Python projects from the BugsInPy curated collection. The results replicate for Python several results known about Java, and shed light on whether Python's peculiarities affect the capabilities of fault localization. The replication package that accompanies this paper includes detailed data about our experiments, as well as the tool FauxPy that we implemented to conduct the study.
翻译:尽管Python作为一种编程语言具有极高的人气,特别是在数据科学程序等新兴领域中,但针对Python的故障定位研究却相对较少。虽然关于C/C++和Java(故障定位研究中最常用的语言)的若干发现可能适用于其他语言,但Python的动态特性及其在实际中的使用方式是否会影响经典故障定位方法的能力,仍是待探究的开放性问题。本文首次在真实世界Python程序和故障上开展了大规模故障定位实证研究。以Zou等人近期对Java故障定位的大规模实证研究为基础,我们针对来自BugsInPy精选数据集中13个开源Python项目的135个故障,评估了四类(基于频谱、基于变异、谓词切换和基于堆栈跟踪)七种知名故障定位技术的有效性(即定位精度)、效率(即运行时性能)及其他特征(如不同实体粒度)。研究结果复现了Java中已知的若干结论,并揭示了Python特性是否会影响故障定位的能力。本文附带的复现包包含实验的详细数据,以及我们为开展本研究所实现的工具FauxPy。