Deep Neural Networks (DNNs) are increasingly being deployed in security-critical and safety-sensitive applications, which makes rigorous testing essential to identify and mitigate model weaknesses. Existing DNN testing approaches explore either the input space or a learned latent space. While latent-space generation can better maintain plausibility than direct input-space mutation, current methods still face a trade-off among exploration controllability, failure diversity, and seed-relative semantic drift. To overcome these limitations, we propose Latte, a black-box testing framework that generates semantically proximate, diverse, and fault-revealing test cases by leveraging the latent space. Specifically, Latte encodes each input seed with a pre-trained VQ-VAE and performs a seed-centered, one-step latent mutation along directions defined by anchors sampled from alternative classes, followed by quantization and decoding back to the input space. This explores local neighborhoods around each seed within the learned latent manifold, resulting in a larger number and broader diversity of oracle-triggering prediction discrepancies under the same budget. We evaluated Latte on 5 datasets and 10 DNN models in single-model and multi-model testing scenarios. Across the evaluated datasets and models, Latte improves fault exposure and behavioral diversity under matched testing budgets. Under the single-model setting, it also maintains low seed-relative semantic drift with respect to the source seeds.
翻译:摘要:深度神经网络(Deep Neural Networks, DNNs)正越来越多地被部署于安全关键型与安全敏感型应用中,这使得严格的测试对于识别并缓解模型缺陷至关重要。现有的DNN测试方法要么探索输入空间,要么探索学习到的潜在空间。尽管潜在空间生成比直接进行输入空间突变能更好地维持合理性,但当前方法在探索可控性、故障多样性及种子相对语义漂移之间仍面临权衡。为克服这些限制,我们提出Latte——一种利用潜在空间生成语义邻近、多样且能揭示缺陷测试用例的黑盒测试框架。具体而言,Latte使用预训练的VQ-VAE对每个输入种子进行编码,并沿从替代类中采样的锚点所定义的向量方向,执行以种子为中心的单步潜在突变,随后进行量化并解码回输入空间。这将在学习到的潜在流形中探索每个种子周围的局部邻域,从而在相同预算下触发预言违反预测偏差的数量更多且多样性更广。我们在单模型与多模型测试场景下,基于5个数据集和10个DNN模型对Latte进行了评估。在匹配测试预算的条件下,Latte在评估的数据集与模型上均提升了缺陷暴露能力与行为多样性。在单模型设置下,它还能保持相对于源种子较低的语义漂移。