Recently, creative generative artificial intelligence software has emerged as a pivotal assistant, enabling users to generate content and seek inspiration rapidly. Text-to-image (T2I) software, being one of the most widely used among them, is used to synthesize images with simple text input by engaging in a cross-modal process. However, despite substantial advancements in several fields, T2I software often encounters defects and erroneous, including omitting focal entities, low image realism, and mismatched text-image information. The cross-modal nature of T2I software makes it challenging for traditional testing methods to detect defects. Lacking test oracles further increases the complexity of testing. To address this deficiency, we propose ACTesting, an Automated Cross-modal Testing Method of Text-to-Image software, the first testing method designed specifically for T2I software. We construct test samples based on entities and relationship triples following the fundamental principle of maintaining consistency in the semantic information to overcome the cross-modal matching challenges. To address the issue of testing oracle scarcity, we first design the metamorphic relation for T2I software and implement three types of mutation operators guided by adaptability density. In the experiment, we conduct ACTesting on four widely-used T2I software. The results show that ACTesting can generate error-revealing tests, reducing the text-image consistency by up to 20% compared with the baseline. We also conduct the ablation study that effectively showcases the efficacy of each mutation operator, based on the proposed metamorphic relation. The results demonstrate that ACTesting can identify abnormal behaviors of T2I software effectively.

翻译：近期，创意生成式人工智能软件已成为关键辅助工具，使用户能够快速生成内容并获取灵感。其中，文本到图像（T2I）软件作为应用最广泛的类型之一，通过跨模态过程将简单文本输入合成为图像。然而，尽管在多个领域取得了显著进展，T2I软件仍常出现缺陷和错误，包括遗漏核心实体、图像真实感不足以及文本-图像信息不匹配等问题。T2I软件的跨模态特性使得传统测试方法难以检测缺陷，而缺乏测试预言进一步增加了测试复杂性。为解决这一不足，我们提出ACTesting——一种面向文本到图像软件的自动化跨模态测试方法，这是首个专门针对T2I软件设计的测试方法。我们基于保持语义信息一致性的基本原则，构建基于实体和关系三元组的测试样本，以克服跨模态匹配难题。针对测试预言稀缺问题，我们首先设计了T2I软件的蜕变关系，并基于适应性密度指导实现了三种类型的变异算子。实验中，我们对四款广泛使用的T2I软件进行了ACTesting测试。结果表明，ACTesting能够生成暴露错误的测试用例，与基线方法相比，文本-图像一致性最多降低20%。我们还基于所提出的蜕变关系进行了消融实验，有效展示了每种变异算子的功效。结果证明，ACTesting能够有效识别T2I软件的异常行为。