In many applications, machine learned (ML) models are required to hold some invariance qualities, such as rotation, size, intensity, and background invariance. Unlike many types of variance, the variants of background scenes cannot be ordered easily, which makes it difficult to analyze the robustness and biases of the models concerned. In this work, we present a technical solution for ordering background scenes according to their semantic proximity to a target image that contains a foreground object being tested. We make use of the results of object recognition as the semantic description of each image, and construct an ontology for storing knowledge about relationships among different objects using association analysis. This ontology enables (i) efficient and meaningful search for background scenes of different semantic distances to a target image, (ii) quantitative control of the distribution and sparsity of the sampled background scenes, and (iii) quality assurance using visual representations of invariance testing results (referred to as variance matrices). In this paper, we also report the training of an ML4ML assessor to evaluate the invariance quality of ML models automatically.

翻译：在许多应用中,机器学习(ML)模型必须保持某些变化性,如旋转、大小、强度和背景差异等。与许多类型的差异不同,背景场景的变异性不能轻易地排列,因此难以分析有关模型的坚固性和偏差。在这项工作中,我们提出了一个技术解决方案,用于根据背景场景的语义接近目标图像,其中含有正在测试的表面物体。我们利用物体识别结果作为每个图像的语义描述,并建立一个本体学来储存关于使用关联分析的不同对象之间关系的知识。这种本体学能够(一) 高效和有意义地搜索与目标图像不同语义距离的背景场景,(二) 对抽样背景场景的分布和宽度进行定量控制,(三) 使用变量测试结果的直观描述(称为差异矩阵)保证质量。在本文中,我们还报告了对ML4ML评估师的培训,以自动评估ML模型的逆差质量。