Due to the increasing sophistication of web attacks, Web Application Firewalls (WAFs) have to be tested and updated regularly to resist the relentless flow of web attacks. In practice, using a brute-force attack to discover vulnerabilities is infeasible due to the wide variety of attack patterns. Thus, various black-box testing techniques have been proposed in the literature. However, these techniques suffer from low efficiency. This paper presents Reinforcement-Learning-Driven and Adaptive Testing (RAT), an automated black-box testing strategy to discover injection vulnerabilities in WAFs. In particular, we focus on SQL injection and Cross-site Scripting, which have been among the top ten vulnerabilities over the past decade. More specifically, RAT clusters similar attack samples together. It then utilizes a reinforcement learning technique combined with a novel adaptive search algorithm to discover almost all bypassing attack patterns efficiently. We compare RAT with three state-of-the-art methods considering their objectives. The experiments show that RAT performs 33.53% and 63.16% on average better than its counterparts in discovering the most possible bypassing payloads and reducing the number of attempts before finding the first bypassing payload when testing well-configured WAFs, respectively.

翻译：由于网络攻击日益复杂化，Web应用防火墙必须定期测试与更新，以抵御持续不断的网络攻击浪潮。实践中，因攻击模式种类繁多，采用暴力攻击手段发现漏洞并不可行。为此，学界已提出多种黑盒测试技术，但这些技术存在效率低下的问题。本文提出强化学习驱动自适应测试（RAT）——一种用于发现WAF注入漏洞的自动化黑盒测试策略。我们重点研究过去十年间始终位列十大漏洞的SQL注入与跨站脚本攻击。具体而言，RAT将相似攻击样本聚类，进而结合强化学习技术与新型自适应搜索算法，高效发现几乎所有的绕过攻击模式。我们针对其目标将RAT与三种先进方法进行对比。实验表明，在测试配置完善的WAF时，RAT在发现最具可能性的绕过载荷方面平均优于对比方法33.53%，在首次发现绕过载荷前的尝试次数减少方面平均优于对比方法63.16%。