Mocking in the context of automated software tests allows testing program units in isolation. Designing realistic interactions between a unit and its environment, and understanding the expected impact of these interactions on the behavior of the unit, are two key challenges that software testers face when developing tests with mocks. In this paper, we propose to monitor an application in production to generate tests that mimic realistic execution scenarios through mocks. Our approach operates in three phases. First, we instrument a set of target methods for which we want to generate tests, as well as the methods that they invoke, which we refer to as mockable method calls. Second, in production, we collect data about the context in which target methods are invoked, as well as the parameters and the returned value for each mockable method call. Third, offline, we analyze the production data to generate test cases with realistic inputs and mock interactions. The approach is automated and implemented in an open-source tool called RICK. We evaluate our approach with 3 real-world, open-source Java applications. RICK monitors the invocation of 128 methods in production across the 3 applications and captures their behavior. Based on this captured data, RICK generates test cases that include realistic initial states and test inputs, mocks, and stubs. The three kinds of mock-based oracles generated by RICK verify the actual interactions between the method and its environment. All the generated test cases are executable, and 52.4% of them successfully mimic the complete execution context of the methods observed in production. The mock-based oracles are effective at detecting regressions within the target methods, complementing each other in their fault-finding ability. We interview 5 developers from the industry who confirm the relevance of using production observations to design mocks and stubs.

翻译：在自动化软件测试中，模拟技术允许对程序单元进行隔离测试。设计单元与其环境之间的真实交互，并理解这些交互对单元行为的预期影响，是软件开发人员在利用模拟对象编写测试时面临的两个关键挑战。本文提出通过监控应用程序在生产环境中的运行状态，生成能够通过模拟对象复现真实执行场景的测试用例。我们的方法分为三个阶段实施：首先，对一组目标方法及其调用的方法（称为可模拟方法调用）进行插桩；其次，在生产环境中收集目标方法调用时的上下文数据，以及每个可模拟方法调用的参数与返回值；最后，在离线状态下分析生产数据，生成具有真实输入和模拟交互的测试用例。该方法已实现自动化，并集成于开源工具RICK中。我们使用3个真实世界的开源Java应用程序对该方法进行评估。RICK在生产环境中监控了这3个应用程序中128个方法的调用过程，并捕获其行为特征。基于捕获的数据，RICK生成了包含真实初始状态、测试输入、模拟对象和桩代码的测试用例。RICK生成的三类基于模拟的断言验证了方法与其环境之间的实际交互。所有生成的测试用例均可执行，其中52.4%成功复现了生产环境中观察到的完整方法执行上下文。基于模拟的断言能有效检测目标方法中的回归问题，并在缺陷发现能力上形成互补。我们对5位行业开发者进行访谈，他们确认了利用生产环境观察数据设计模拟对象和桩代码的实用价值。