Remote Procedure Call (RPC) is a communication protocol to support client-server interactions among services over a network. RPC is widely applied in industry for building large-scale distributed systems, such as Microservices. Modern RPC frameworks include for example Thrift, gRPC, SOFARPC and Dubbo. Testing such systems is very challenging, due to the complexity of distributed systems and various RPC frameworks the system could employ. To the best of our knowledge, there does not exist any tool or solution that could enable automated testing of modern RPC-based services. To fill this gap, in this paper we propose the first approach in the literature, together with an open-source tool, for white-box fuzzing modern RPC-based APIs with search. To assess our novel approach, we conducted an empirical study with two artificial and four industrial APIs selected by our industrial partner. The tool has been integrated into a real industrial pipeline, and could be applied to real industrial development process for fuzzing RPC-based APIs. To further demonstrate its effectiveness and application in industrial settings, we also report results of employing our tool for fuzzing another 50 industrial APIs autonomously conducted by our industrial partner in their testing processes. Results show that our novel approach is capable of enabling automated test case generation for industrial RPC-based APIs (i.e., two artificial and 54 industrial). We also compared with a simple grey-box technique and existing manually written tests. Our white-box solution achieves significant improvements on code coverage. Regarding fault detection, by conducting a careful review with our industrial partner of the tests generated by our novel approach in the selected four industrial APIs, a total of 41 real faults were identified, which have now been fixed. Another 8,377 detected faults are currently under investigation.
翻译:远程过程调用(RPC)是一种支持网络中服务间客户端-服务器交互的通信协议。RPC在构建大规模分布式系统(如微服务)的工业场景中应用广泛。现代RPC框架包括Thrift、gRPC、SOFARPC和Dubbo等。由于分布式系统的复杂性及系统可能采用的不同RPC框架,对此类系统进行测试极具挑战性。据我们所知,目前尚无任何工具或解决方案能够实现现代RPC服务的自动化测试。为填补这一空白,本文首次提出基于搜索的现代RPC API白盒模糊测试方法及相应的开源工具。为评估该创新方法,我们开展了包含两个人工API和四个由工业合作伙伴选定的工业API的实证研究。该工具已集成到实际工业流水线中,可应用于真实工业开发流程进行RPC API模糊测试。为进一步验证其在工业环境中的有效性,我们报告了工业合作伙伴在测试流程中自主运用该工具对另外50个工业API进行模糊测试的结果。结果表明,该创新方法能够为工业RPC API(即两个人工API和54个工业API)自动生成测试用例。我们将该方法与简单灰盒技术及现有手工测试进行对比:白盒方案在代码覆盖率上取得显著提升。关于故障检测,通过与工业合作伙伴对选定四个工业API中由新方法生成的测试用例进行仔细审查,共识别出41个真实故障(现已修复),另有8,377个检测到的故障正在调查中。