We show the sup-norm convergence of deep neural network estimators with a novel adversarial training scheme. For the nonparametric regression problem, it has been shown that an estimator using deep neural networks can achieve better performances in the sense of the $L2$-norm. In contrast, it is difficult for the neural estimator with least-squares to achieve the sup-norm convergence, due to the deep structure of neural network models. In this study, we develop an adversarial training scheme and investigate the sup-norm convergence of deep neural network estimators. First, we find that ordinary adversarial training makes neural estimators inconsistent. Second, we show that a deep neural network estimator achieves the optimal rate in the sup-norm sense by the proposed adversarial training with correction. We extend our adversarial training to general setups of a loss function and a data-generating function. Our experiments support the theoretical findings.

翻译：我们展示了采用新型对抗训练方案的深度神经网络估计器具有上确界范数收敛性。对于非参数回归问题，已有研究表明，深度神经网络估计器在$L2$-范数意义下能够获得更优性能。然而，由于神经网络模型的深层结构，基于最小二乘法的神经估计器难以实现上确界范数收敛。本研究开发了一种对抗训练方案，并探讨了深度神经网络估计器的上确界范数收敛性。首先，我们发现普通对抗训练会导致神经估计器的不一致性。其次，我们证明通过所提出的带修正的对抗训练，深度神经网络估计器能在上确界范数意义上达到最优收敛速度。我们将该对抗训练方案推广至损失函数与数据生成函数的一般设定。实验结果支持了上述理论发现。