Existing certified training methods can only train models to be robust against a certain perturbation type (e.g. $l_\infty$ or $l_2$). However, an $l_\infty$ certifiably robust model may not be certifiably robust against $l_2$ perturbation (and vice versa) and also has low robustness against other perturbations (e.g. geometric transformation). To this end, we propose the first multi-norm certified training framework \textbf{CURE}, consisting of a new $l_2$ deterministic certified training defense and several multi-norm certified training methods, to attain better \emph{union robustness} when training from scratch or fine-tuning a pre-trained certified model. Further, we devise bound alignment and connect natural training with certified training for better union robustness. Compared with SOTA certified training, \textbf{CURE} improves union robustness up to $22.8\%$ on MNIST, $23.9\%$ on CIFAR-10, and $8.0\%$ on TinyImagenet. Further, it leads to better generalization on a diverse set of challenging unseen geometric perturbations, up to $6.8\%$ on CIFAR-10. Overall, our contributions pave a path towards \textit{universal certified robustness}.

翻译：现有的认证训练方法仅能训练模型对特定类型的扰动（例如$l_\infty$或$l_2$）具有鲁棒性。然而，一个对$l_\infty$扰动具备认证鲁棒性的模型，可能无法对$l_2$扰动具备认证鲁棒性（反之亦然），并且对其他类型的扰动（例如几何变换）也表现出较低的鲁棒性。为此，我们提出了首个多范数认证训练框架 \textbf{CURE}。该框架包含一种新的$l_2$确定性认证训练防御方法以及若干多范数认证训练方法，旨在从零开始训练或微调预训练认证模型时，获得更好的 \emph{并集鲁棒性}。此外，我们设计了边界对齐方法，并将自然训练与认证训练联系起来，以进一步提升并集鲁棒性。与最先进的认证训练方法相比，\textbf{CURE} 在MNIST数据集上将并集鲁棒性提升了高达$22.8\%$，在CIFAR-10上提升了$23.9\%$，在TinyImagenet上提升了$8.0\%$。此外，它在多种具有挑战性的未见几何扰动上实现了更好的泛化能力，在CIFAR-10上提升高达$6.8\%$。总体而言，我们的贡献为迈向 \textit{通用认证鲁棒性} 铺平了道路。