Mocking in the context of automated software tests allows testing program units in isolation. Designing realistic interactions between a unit and its environment, and understanding the expected impact of these interactions on the behavior of the unit, are two key challenges that software testers face when developing tests with mocks. In this paper, we propose to monitor an application in production to generate tests that mimic realistic execution scenarios through mocks. Our approach operates in three phases. First, we instrument a set of target methods for which we want to generate tests, as well as the methods that they invoke, which we refer to as mockable method calls. Second, in production, we collect data about the context in which target methods are invoked, as well as the parameters and the returned value for each mockable method call. Third, offline, we analyze the production data to generate test cases with realistic inputs and mock interactions. The approach is automated and implemented in an open-source tool called RICK. We evaluate our approach with 3 real-world, open-source Java applications. RICK monitors the invocation of 128 methods in production across the 3 applications and captures their behavior. Based on this captured data, RICK generates test cases that include realistic initial states and test inputs, mocks, and stubs. The three kinds of mock-based oracles generated by RICK verify the actual interactions between the method and its environment. All the generated test cases are executable, and 52.4% of them successfully mimic the complete execution context of the methods observed in production. The mock-based oracles are effective at detecting regressions within the target methods, complementing each other in their fault-finding ability. We interview 5 developers from the industry who confirm the relevance of using production observations to design mocks and stubs.

翻译：在自动化软件测试中，模拟对象（Mock）允许对程序单元进行隔离测试。设计测试单元与其环境之间的真实交互，并理解这些交互对单元行为的预期影响，是开发者在运用模拟对象编写测试时面临的两大核心挑战。本文提出通过监控生产环境中的应用程序，利用模拟对象生成能够模拟真实执行场景的测试用例。该方法分为三个阶段：首先，对目标方法（即需要生成测试的方法）及其调用的方法（称为可模拟方法调用）进行插桩；其次，在生产环境中收集目标方法被调用的上下文信息，以及每个可模拟方法调用的参数与返回值；最后，离线分析生产数据，生成包含真实输入与模拟交互的测试用例。该方法已实现自动化，并开发为开源工具RICK。我们使用3个真实世界中的开源Java应用程序进行评估。RICK在3个应用程序的生产环境中监控了128个方法的调用行为，并基于捕获的数据生成测试用例，这些用例包含真实初始状态、测试输入、模拟对象与桩代码（Stubs）。RICK生成的三种基于模拟的预言（Mock-based Oracles）可验证方法与其环境之间的实际交互。所有生成的测试用例均可执行，其中52.4%成功模拟了生产环境中观察到的完整执行上下文。这些基于模拟的预言在检测目标方法内的回归缺陷方面效果显著，且彼此在故障发现能力上互补。我们采访了5位工业界开发者，他们确认了利用生产观测数据设计模拟对象与桩代码的实际价值。