Deep Neural Networks (DNNs) have been widely employed across various domains, including safety-critical systems, necessitating comprehensive testing to ensure their reliability. Although numerous DNN model testing methods have been proposed to generate adversarial samples that are capable of revealing faults, existing methods typically perturb samples in the input space and then mutate these based on feedback from the DNN model. These methods often result in test samples that are not realistic and with low-probability reveal faults. To address these limitations, we propose a black-box DNN test input generation method, ARGUS, to generate realistic, diverse, and fault-revealing test inputs. ARGUS first compresses samples into a continuous latent space and then perturbs the original samples by interpolating these with samples of different classes. Subsequently, we employ a vector quantizer and decoder to reconstruct adversarial samples back into the input space. Additionally, we employ discriminators both in the latent space and in the input space to ensure the realism of the generated samples. Evaluation of ARGUS in comparison with state-of-the-art black-box testing and white-box testing methods, shows that ARGUS excels in generating realistic and diverse adversarial samples relative to the target dataset, and ARGUS successfully perturbs all original samples and achieves up to 4 times higher error rate than the best baseline method. Furthermore, using these adversarial samples for model retraining can improve model classification accuracy.

翻译：深度神经网络（DNN）已广泛应用于包括安全关键系统在内的多个领域，这要求对其进行全面测试以确保其可靠性。尽管已有许多DNN模型测试方法被提出，用于生成能够揭示错误的对抗样本，但现有方法通常在输入空间中对样本进行扰动，然后根据DNN模型的反馈对这些样本进行变异。这些方法往往导致生成的测试样本不真实，且揭示错误的概率较低。为解决这些局限性，我们提出了一种黑盒DNN测试输入生成方法——ARGUS，以生成真实、多样且能揭示错误的测试输入。ARGUS首先将样本压缩到一个连续的潜在空间中，然后通过将原始样本与不同类别的样本进行插值来扰动原始样本。随后，我们采用向量量化器和解码器将对抗样本重构回输入空间。此外，我们在潜在空间和输入空间中均使用了判别器，以确保生成样本的真实性。通过将ARGUS与最先进的黑盒测试和白盒测试方法进行比较评估，结果表明，相对于目标数据集，ARGUS在生成真实且多样的对抗样本方面表现优异，并且成功扰动了所有原始样本，其错误率最高可达最佳基线方法的4倍。此外，使用这些对抗样本进行模型重训练可以提高模型的分类准确率。