Many test coverage metrics have been proposed to measure the Deep Neural Network (DNN) testing effectiveness, including structural coverage and non-structural coverage. These test coverage metrics are proposed based on the fundamental assumption: they are correlated with test effectiveness. However, the fundamental assumption is still not validated sufficiently and reasonably, which brings question on the usefulness of DNN test coverage. This paper conducted a revisiting study on the existing DNN test coverage from the test effectiveness perspective, to effectively validate the fundamental assumption. Here, we carefully considered the diversity of subjects, three test effectiveness criteria, and both typical and state-of-the-art test coverage metrics. Different from all the existing studies that deliver negative conclusions on the usefulness of existing DNN test coverage, we identified some positive conclusions on their usefulness from the test effectiveness perspective. In particular, we found the complementary relationship between structural and non-structural coverage and identified the practical usage scenarios and promising research directions for these existing test coverage metrics.

翻译：许多测试覆盖度量指标已被提出用于衡量深度神经网络（DNN）的测试有效性，包括结构覆盖度和非结构覆盖度。这些测试覆盖度量指标基于一个基本假设：它们与测试有效性相关。然而，该基本假设尚未得到充分且合理的验证，这导致了对DNN测试覆盖率实用性的质疑。本文从测试有效性角度对现有DNN测试覆盖率进行了重新审视研究，以有效验证这一基本假设。在此过程中，我们谨慎考虑了研究主体的多样性、三种测试有效性准则，以及典型和最新的测试覆盖度量指标。与所有现有研究对现有DNN测试覆盖率实用性得出负面结论不同，我们从测试有效性角度发现了其积极结论。具体而言，我们发现了结构覆盖度和非结构覆盖度之间的互补关系，并明确了这些现有测试覆盖度量指标的实际使用场景和有前景的研究方向。