The theoretical structure of deep neural network (DNN) has been clarified gradually. Imaizumi-Fukumizu (2019) and Suzuki (2019) clarified that the learning ability of DNN is superior to the previous theories when the target function is non-smooth functions. However, as far as the author is aware, none of the numerous works to date attempted to mathematically investigate what kind of DNN architectures really induce pointwise convergence of gradient descent (without any statistical argument), and this attempt seems to be closer to the practical DNNs. In this paper we restrict target functions to non-smooth indicator functions, and construct a deep neural network inducing pointwise convergence provided by mini-batch gradient descent process in ReLU-DNN.

翻译：深度神经网络（DNN）的理论结构已逐步清晰。Imaizumi-Fukumizu（2019）与Suzuki（2019）阐明，当目标函数为非光滑函数时，DNN的学习能力优于以往理论。然而，据作者所知，迄今为止众多研究中尚无工作尝试从数学角度探究何种DNN架构真正能引发梯度下降（不包含任何统计论证）的点态收敛，而这一尝试似乎更贴近实际DNN。本文将目标函数限制为非光滑的示性函数，构建了一种在ReLU-DNN中由小批量梯度下降过程引发点态收敛的深度神经网络。