This work focuses on the problem of hyper-parameter tuning (HPT) for robust (i.e., adversarially trained) models, with the twofold goal of i) establishing which additional HPs are relevant to tune in adversarial settings, and ii) reducing the cost of HPT for robust models. We pursue the first goal via an extensive experimental study based on 3 recent models widely adopted in the prior literature on adversarial robustness. Our findings show that the complexity of the HPT problem, already notoriously expensive, is exacerbated in adversarial settings due to two main reasons: i) the need of tuning additional HPs which balance standard and adversarial training; ii) the need of tuning the HPs of the standard and adversarial training phases independently. Fortunately, we also identify new opportunities to reduce the cost of HPT for robust models. Specifically, we propose to leverage cheap adversarial training methods to obtain inexpensive, yet highly correlated, estimations of the quality achievable using state-of-the-art methods (PGD). We show that, by exploiting this novel idea in conjunction with a recent multi-fidelity optimizer (taKG), the efficiency of the HPT process can be significantly enhanced.

翻译：本文聚焦于鲁棒（即对抗训练）模型的超参数调优（HPT）问题，旨在实现双重目标：i）确定对抗场景下需要额外调优的相关超参数，ii）降低鲁棒模型超参数调优的成本。针对第一个目标，我们基于先前对抗鲁棒性文献中广泛采用的三种最新模型开展了大量实验研究。结果表明，本已复杂度高昂的超参数调优问题在对抗场景中进一步加剧，主要原因有二：i）需要额外调优用于平衡标准训练与对抗训练的超参数；ii）需要独立调优标准训练阶段与对抗训练阶段的超参数。幸运的是，我们也识别出降低鲁棒模型超参数调优成本的新机遇。具体而言，我们提出利用廉价对抗训练方法，获得与使用最先进方法（PGD）所能达到的质量高度相关且成本低廉的估计值。研究表明，通过将这一新颖思想与近期提出的多保真优化器（taKG）相结合，可显著提升超参数调优过程的效率。