Fuzzing is an effective bug-finding technique but it struggles with complex systems like JavaScript engines that demand precise grammatical input. Recently, researchers have adopted language models for context-aware mutation in fuzzing to address this problem. However, existing techniques are limited in utilizing coverage guidance for fuzzing, which is rather performed in a black-box manner. This paper presents a novel technique called CovRL (Coverage-guided Reinforcement Learning) that combines Large Language Models (LLMs) with reinforcement learning from coverage feedback. Our fuzzer, CovRL-Fuzz, integrates coverage feedback directly into the LLM by leveraging the Term Frequency-Inverse Document Frequency (TF-IDF) method to construct a weighted coverage map. This map is key in calculating the fuzzing reward, which is then applied to the LLM-based mutator through reinforcement learning. CovRL-Fuzz, through this approach, enables the generation of test cases that are more likely to discover new coverage areas, thus improving vulnerability detection while minimizing syntax and semantic errors, all without needing extra post-processing. Our evaluation results indicate that CovRL-Fuzz outperforms the state-of-the-art fuzzers in terms of code coverage and bug-finding capabilities: CovRL-Fuzz identified 48 real-world security-related bugs in the latest JavaScript engines, including 39 previously unknown vulnerabilities and 11 CVEs.

翻译：摘要：模糊测试是一种高效的漏洞发现技术，但在处理需要精确语法输入的复杂系统（如JavaScript引擎）时存在困难。近年来，研究人员采用语言模型进行上下文感知的模糊测试变异以解决此问题。然而，现有技术在利用覆盖引导进行模糊测试方面存在局限性，往往以黑盒方式运行。本文提出一种名为CovRL（覆盖引导强化学习）的新型技术，将大语言模型与基于覆盖反馈的强化学习相结合。我们的模糊测试工具CovRL-Fuzz通过利用词频-逆文档频率方法构建加权覆盖图，将覆盖反馈直接集成到大语言模型中。该覆盖图是计算模糊测试奖励的关键，进而通过强化学习应用于基于大语言模型的变异器。通过这种方法，CovRL-Fuzz能够生成更可能发现新覆盖区域的测试用例，从而提高漏洞检测能力，同时最大限度减少语法和语义错误，且无需额外后处理。评估结果表明，CovRL-Fuzz在代码覆盖率和漏洞发现能力方面均优于现有最先进的模糊测试工具：CovRL-Fuzz在最新JavaScript引擎中发现了48个真实世界安全相关漏洞，包括39个此前未知的漏洞和11个CVE编号。