Fuzzing has contributed to automatically identifying bugs and vulnerabilities in the software testing field. Although it can efficiently generate crashing inputs, these inputs are usually analyzed manually. Several root cause analysis (RCA) techniques have been proposed to automatically analyze the root causes of crashes to mitigate this cost. However, outstanding challenges for realizing more elaborate RCA techniques remain unknown owing to the lack of extensive evaluation methods over existing techniques. With this problem in mind, we developed an end-to-end benchmarking platform, RCABench, that can evaluate RCA techniques for various targeted programs in a detailed and comprehensive manner. Our experiments with RCABench indicated that the evaluations in previous studies were not enough to fully support their claims. Moreover, this platform can be leveraged to evaluate emerging RCA techniques by comparing them with existing techniques.

翻译：模糊测试在软件测试领域已有效促进漏洞和缺陷的自动识别。尽管该技术能高效生成程序崩溃输入，但这些输入通常需要人工分析。为降低这一成本，研究者提出了多种根因分析（RCA）技术以自动识别崩溃根因。然而，由于缺乏对现有技术的全面评估方法，实现更精细化的RCA技术仍面临突出挑战。针对该问题，我们开发了端到端基准测试平台RCABench，可对面向不同目标程序的RCA技术进行细致且全面的评估。基于RCABench的实验表明，以往研究中的评估不足以完全支撑其结论。此外，该平台还可通过比较现有技术来评估新兴的RCA技术。