We study the implicit bias of optimization in robust empirical risk minimization (robust ERM) and its connection with robust generalization. In classification settings under adversarial perturbations with linear models, we study what type of regularization should ideally be applied for a given perturbation set to improve (robust) generalization. We then show that the implicit bias of optimization in robust ERM can significantly affect the robustness of the model and identify two ways this can happen; either through the optimization algorithm or the architecture. We verify our predictions in simulations with synthetic data and experimentally study the importance of implicit bias in robust ERM with deep neural networks.

翻译：本研究探讨了鲁棒经验风险最小化（鲁棒ERM）中优化过程的隐式偏差及其与鲁棒泛化的关联。在线性模型对抗扰动下的分类场景中，我们研究了针对给定扰动集应施加何种理想的正则化以提升（鲁棒）泛化性能。随后，我们证明鲁棒ERM中优化过程的隐式偏差会显著影响模型的鲁棒性，并识别出两种可能的发生途径：通过优化算法或模型架构。我们通过合成数据仿真验证了理论预测，并基于深度神经网络对鲁棒ERM中隐式偏差的重要性进行了实验研究。