We study gradient-based regularization methods for neural networks. We mainly focus on two regularization methods: the total variation and the Tikhonov regularization. Applying these methods is equivalent to using neural networks to solve some partial differential equations, mostly in high dimensions in practical applications. In this work, we introduce a general framework to analyze the generalization error of regularized networks. The error estimate relies on two assumptions on the approximation error and the quadrature error. Moreover, we conduct some experiments on the image classification tasks to show that gradient-based methods can significantly improve the generalization ability and adversarial robustness of neural networks. A graphical extension of the gradient-based methods are also considered in the experiments.

翻译：我们研究神经网络的基于梯度的正规化方法,我们主要侧重于两种正规化方法:总变异和Tikhonov的正规化。应用这些方法相当于使用神经网络解决部分差异方程式,大多在实际应用中具有很高的层面。在这项工作中,我们引入了分析正规化网络一般化错误的一般框架。错误估计依据了近似误差和二次误差的两个假设。此外,我们对图像分类任务进行了一些实验,以表明基于梯度的方法可以显著提高神经网络的通用能力和对抗性强度。实验中也考虑了基于梯度的方法的图形扩展。