Empirical robustness evaluation (RE) of deep learning models against adversarial perturbations entails solving nontrivial constrained optimization problems. Existing numerical algorithms that are commonly used to solve them in practice predominantly rely on projected gradient, and mostly handle perturbations modeled by the $\ell_1$, $\ell_2$ and $\ell_\infty$ distances. In this paper, we introduce a novel algorithmic framework that blends a general-purpose constrained-optimization solver PyGRANSO with Constraint Folding (PWCF), which can add more reliability and generality to the state-of-the-art RE packages, e.g., AutoAttack. Regarding reliability, PWCF provides solutions with stationarity measures and feasibility tests to assess the solution quality. For generality, PWCF can handle perturbation models that are typically inaccessible to the existing projected gradient methods; the main requirement is the distance metric to be almost everywhere differentiable. Taking advantage of PWCF and other existing numerical algorithms, we further explore the distinct patterns in the solutions found for solving these optimization problems using various combinations of losses, perturbation models, and optimization algorithms. We then discuss the implications of these patterns on the current robustness evaluation and adversarial training.

翻译：深度学习模型对抗扰动的经验鲁棒性评估（RE）需要求解非平凡的约束优化问题。目前实践中常用的数值算法主要依赖于投影梯度法，且大多处理由$\ell_1$、$\ell_2$和$\ell_\infty$距离建模的扰动。本文提出一种新颖的算法框架，将通用约束优化求解器PyGRANSO与约束折叠（PWCF）相结合，可为现有最先进的鲁棒性评估工具包（如AutoAttack）增加更高的可靠性和通用性。在可靠性方面，PWCF提供具备平稳性度量和可行性检验的解决方案，用于评估解的质量。在通用性方面，PWCF可处理现有投影梯度法通常无法处理的扰动模型，其主要要求是距离度量需几乎处处可微。借助PWCF及其他现有数值算法，我们进一步探索了使用不同损失函数、扰动模型和优化算法组合求解这些优化问题时发现的解的独特模式。最后，我们讨论了这些模式对当前鲁棒性评估和对抗训练的意义。