Fuzzing has contributed to automatically identifying bugs and vulnerabilities in the software testing field. Although it can efficiently generate crashing inputs, these inputs are usually analyzed manually. Several root cause analysis (RCA) techniques have been proposed to automatically analyze the root causes of crashes to mitigate this cost. However, outstanding challenges for realizing more elaborate RCA techniques remain unknown owing to the lack of extensive evaluation methods over existing techniques. With this problem in mind, we developed an end-to-end benchmarking platform, RCABench, that can evaluate RCA techniques for various targeted programs in a detailed and comprehensive manner. Our experiments with RCABench indicated that the evaluations in previous studies were not enough to fully support their claims. Moreover, this platform can be leveraged to evaluate emerging RCA techniques by comparing them with existing techniques.

翻译：模糊测试在软件测试领域已有效助力自动化识别缺陷与漏洞。尽管它能高效生成崩溃触发输入，但这些输入通常需人工分析。为降低此成本，研究者提出了多种根因分析（RCA）技术来自动解析崩溃的根本原因。然而，由于缺乏对现有技术进行全面评估的方法，实现更精密的RCA技术仍面临未知的突出挑战。针对该问题，我们开发了端到端基准测试平台RCABench，能以细致全面的方式评估面向多种目标程序的RCA技术。基于RCABench的实验表明，已有研究中的评估不足以充分支撑其结论。此外，该平台可通过对比新兴技术与现有技术，用于评估新兴RCA技术。