Over the past decade, the automated generation of test inputs has made significant advances. Modern fuzzers and test generators easily produce complex input formats that do systematically cover the input and execution space. Testing _protocols_, though, has remained a frontier for automated testing, as a test generator has to _interact_ with the program under test, producing messages that conform to the current state of the system. In this paper, we introduce _language-based protocol testing_, the first approach to specify, automatically test, and systematically cover the full state and input space of protocol implementations. We specify protocols as _interaction grammars_ -- an extension of context-free grammars that tag each message element with the communication party that is in charge of producing it. Interaction grammars embed classical state models by unifying states, messages, and transitions all into nonterminals, and can be used for _producing_ interactions as well as _parsing_ them, making them ideally suited for testing protocols. Additional _constraints_ over grammar elements allow us to specify and test _semantic features_ such as binary message formats, checksums, encodings, and the many ways that message features induce states and vice versa. To evaluate the effectiveness of language-based protocol testing, we have implemented it as part of the FANDANGO test generator. We specify several protocols as interaction grammars, including features such as human-readable interactions (SMTP), bit-level encodings (DNS), and dynamic port assignments (FTP), and use them to test the corresponding protocol implementations. By systematically covering the interaction grammar and solving the associated constraints, FANDANGO achieves comprehensive coverage of the protocol interactions, resulting in high code coverage and a thorough assessment of the program under test.

翻译：过去十年间，自动化测试输入生成技术取得了显著进展。现代模糊测试工具和测试生成器能够轻松生成复杂的输入格式，系统性地覆盖输入与执行空间。然而，协议测试仍是自动化测试的前沿领域，因为测试生成器必须与被测程序进行交互，生成符合系统当前状态的消息。本文提出基于语言的协议测试方法，这是首个能够对协议实现的全状态与输入空间进行规范描述、自动化测试及系统性覆盖的技术方案。我们将协议规范定义为交互语法——一种上下文无关文法的扩展形式，通过为每个消息元素标注负责生成它的通信方来建模协议。交互语法通过将状态、消息和转移统一表示为非终结符，从而嵌入了经典状态模型，既能用于生成交互序列，也能解析交互过程，使其成为协议测试的理想工具。通过对语法元素施加额外约束，我们能够规范并测试包括二进制消息格式、校验和、编码方式以及消息特征与状态间相互影响机制在内的语义特性。为评估基于语言的协议测试效果，我们在FANDANGO测试生成器中实现了该框架。我们使用交互语法规范了多个协议特性，包括人类可读交互（SMTP）、比特级编码（DNS）和动态端口分配（FTP），并据此测试了相应的协议实现。通过系统覆盖交互语法空间并求解相关约束，FANDANGO实现了对协议交互的全面覆盖，从而获得高代码覆盖率，并对被测程序进行了深入评估。