We propose to study and promote the robustness of a model as per its performance through the interpolation of training data distributions. Specifically, (1) we augment the data by finding the worst-case Wasserstein barycenter on the geodesic connecting subpopulation distributions of different categories. (2) We regularize the model for smoother performance on the continuous geodesic path connecting subpopulation distributions. (3) Additionally, we provide a theoretical guarantee of robustness improvement and investigate how the geodesic location and the sample size contribute, respectively. Experimental validations of the proposed strategy on \textit{four} datasets, including CIFAR-100 and ImageNet, establish the efficacy of our method, e.g., our method improves the baselines' certifiable robustness on CIFAR10 up to $7.7\%$, with $16.8\%$ on empirical robustness on CIFAR-100. Our work provides a new perspective of model robustness through the lens of Wasserstein geodesic-based interpolation with a practical off-the-shelf strategy that can be combined with existing robust training methods.
翻译:我们提出通过训练数据分布的插值来研究并提升模型在性能方面的鲁棒性。具体而言:(1)我们通过寻找连接不同类别子群体分布的测地线上的最坏情况Wasserstein重心来增强数据;(2)我们对模型进行正则化,使其在连接子群体分布的连续测地线路径上性能更加平滑;(3)此外,我们提供了鲁棒性提升的理论保证,并分别探究了测地线位置与样本量的贡献。在包括CIFAR-100和ImageNet在内的四个数据集上对所提策略的实验验证确立了其有效性——例如,我们的方法将CIFAR10上基线模型的可认证鲁棒性提升了高达7.7%,并将CIFAR-100上的经验鲁棒性提升了16.8%。我们的工作通过Wasserstein测地线插值的视角为模型鲁棒性提供了新思路,同时提供了一种实用的即用型策略,可与现有鲁棒训练方法相结合。