In the context of deep learning research, where model introductions continually occur, the need for effective and efficient evaluation remains paramount. Existing methods often emphasize accuracy metrics, overlooking stability. To address this, the paper introduces the Accuracy-Stability Index (ASI), a quantitative measure incorporating both accuracy and stability for assessing deep learning models. Experimental results demonstrate the application of ASI, and a 3D surface model is presented for visualizing ASI, mean accuracy, and coefficient of variation. This paper addresses the important issue of quantitative benchmarking metrics for deep learning models, providing a new approach for accurately evaluating accuracy and stability of deep learning models. The paper concludes with discussions on potential weaknesses and outlines future research directions.
翻译:在深度学习研究中,随着新模型的不断涌现,高效且有效的评估方法至关重要。现有方法常侧重于精度指标,而忽视了稳定性。为解决这一问题,本文提出了精度-稳定性指标(ASI),这是一种综合考量精度与稳定性的定量评估指标,用于评估深度学习模型。实验结果展示了ASI的应用,并构建了三维曲面模型以可视化ASI、平均精度及变异系数。本文针对深度学习模型定量基准评估指标这一重要问题,为准确评估深度学习模型的精度与稳定性提供了新方法。最后,本文讨论了该方法的潜在不足并展望了未来研究方向。