While ChatGPT is a well-known artificial intelligence chatbot being used to answer human's questions, one may want to discover its potential in advancing software testing. We examine the capability of ChatGPT in advancing the intelligence of software testing through a case study on metamorphic testing (MT), a state-of-the-art software testing technique. We ask ChatGPT to generate candidates of metamorphic relations (MRs), which are basically necessary properties of the object program and which traditionally require human intelligence to identify. These MR candidates are then evaluated in terms of correctness by domain experts. We show that ChatGPT can be used to generate new correct MRs to test several software systems. Having said that, the majority of MR candidates are either defined vaguely or incorrect, especially for systems that have never been tested with MT. ChatGPT can be used to advance software testing intelligence by proposing MR candidates that can be later adopted for implementing tests; but human intelligence should still inevitably be involved to justify and rectify their correctness.

翻译：虽然ChatGPT作为知名人工智能聊天机器人被用于回答人类问题，但人们可能希望发掘其在推进软件测试方面的潜力。我们通过一项关于蜕变测试（MT）——一种前沿软件测试技术的案例研究，检验了ChatGPT提升软件测试智能性的能力。我们让ChatGPT生成蜕变关系（MRs）候选，这些关系本质上是目标程序的必然属性，传统上需要人类智能才能识别。随后由领域专家评估这些MR候选的正确性。研究表明，ChatGPT可用于生成正确的新MR来测试多个软件系统。然而，大多数MR候选要么定义模糊要么不正确，尤其是在从未使用MT测试的系统上。ChatGPT可通过提出可供后续实施测试的MR候选来推动软件测试智能化，但人类智能仍需不可避免地介入以验证和纠正其正确性。