This paper adds to the fundamental body of work on benchmarking the robustness of deep learning (DL) classifiers. We innovate a new benchmarking methodology to evaluate robustness of DL classifiers. Also, we introduce a new four-quadrant statistical visualization tool, including minimum accuracy, maximum accuracy, mean accuracy, and coefficient of variation, for benchmarking robustness of DL classifiers. To measure robust DL classifiers, we created a comprehensive 69 benchmarking image set, including a clean set, sets with single factor perturbations, and sets with two-factor perturbation conditions. After collecting experimental results, we first report that using two-factor perturbed images improves both robustness and accuracy of DL classifiers. The two-factor perturbation includes (1) two digital perturbations (salt & pepper noise and Gaussian noise) applied in both sequences, and (2) one digital perturbation (salt & pepper noise) and a geometric perturbation (rotation) applied in both sequences. All source codes, related image sets, and preliminary data, figures are shared on a GitHub website to support future academic research and industry projects. The web resources locate at https://github.com/caperock/robustai

翻译：本文深化了深度学习分类器鲁棒性基准测试的基础研究，提出了一种创新的基准测试方法来评估深度学习分类器的鲁棒性。我们引入了一种新的四象限统计可视化工具，包括最小准确率、最大准确率、平均准确率和变异系数，用于基准测试深度学习分类器的鲁棒性。为测量鲁棒深度学习分类器，我们构建了包含69个基准图像的全面图像集，涵盖干净图像集、单因素扰动图像集及双因素扰动条件图像集。通过实验结果分析，我们首次报告使用双因素扰动图像可同时提升深度学习分类器的鲁棒性与准确率。双因素扰动包括：(1)两种数字扰动（椒盐噪声和高斯噪声）按两种序列组合应用；(2)一种数字扰动（椒盐噪声）与几何扰动（旋转）按两种序列组合应用。所有源代码、相关图像集、初步数据及图表均已共享至GitHub网站（https://github.com/caperock/robustai），以支持未来学术研究与工业项目。