As the demand for verifiability and testability of neural networks continues to rise, an increasing number of methods for generating test sets are being developed. However, each of these techniques tends to emphasize specific testing aspects and can be quite time-consuming. A straightforward solution to mitigate this issue is to transfer test sets between some benchmarked models and a new model under test, based on a desirable property one wishes to transfer. This paper introduces GIST (Generated Inputs Sets Transferability), a novel approach for the efficient transfer of test sets among Deep Learning models. Given a property of interest that a user wishes to transfer (e.g., coverage criterion), GIST enables the selection of good test sets from the point of view of this property among available ones from a benchmark. We empirically evaluate GIST on fault types coverage property with two modalities and different test set generation procedures to demonstrate the approach's feasibility. Experimental results show that GIST can select an effective test set for the given property to transfer it to the model under test. Our results suggest that GIST could be applied to transfer other properties and could generalize to different test sets' generation procedures and modalities
翻译:随着神经网络可验证性与可测试性需求的持续增长,越来越多的测试集生成方法被开发出来。然而,这些技术往往侧重于特定的测试方面,且可能相当耗时。缓解这一问题的一个直接解决方案是,基于希望迁移的理想属性,在基准模型与待测试的新模型之间迁移测试集。本文提出GIST(生成输入集迁移性)——一种在深度学习模型中高效迁移测试集的新方法。给定用户希望迁移的目标属性(例如覆盖率准则),GIST能够从基准模型已有的测试集中,选取在该属性上表现良好的测试集。我们通过两种模态的故障类型覆盖属性及不同的测试集生成程序,对GIST进行实证评估以证明该方法的可行性。实验结果表明,GIST可针对给定属性选取有效的测试集,并将其迁移至待测模型。研究结果表明,GIST可应用于其他属性的迁移,并可能推广至不同测试集的生成程序与模态。