The theoretical structure of deep neural network (DNN) has been clarified gradually. Imaizumi-Fukumizu (2019) and Suzuki (2019) clarified that the learning ability of DNN is superior to the previous theories when the target function is non-smooth functions. However, as far as the author is aware, none of the numerous works to date attempted to mathematically investigate what kind of DNN architectures really induce pointwise convergence of gradient descent (without any statistical argument), and this attempt seems to be closer to the practical DNNs. In this paper we restrict target functions to non-smooth indicator functions, and construct a deep neural network inducing pointwise convergence provided by gradient descent process in ReLU-DNN. The DNN has a sparse and a special shape, with certain variable transformations.

翻译：深度神经网络（DNN）的理论结构已逐渐明晰。Imaizumi-Fukumizu（2019）与Suzuki（2019）阐明，当目标函数为非光滑函数时，DNN的学习能力优于先前理论。然而，据作者所知，迄今为止众多研究均未尝试从数学上探究何种DNN架构能真正引发梯度下降的点态收敛（无需任何统计论证），而这一尝试似乎更贴近实际DNN的应用。本文将目标函数限制为非光滑指示函数，并构造了一种在ReLU-DNN中通过梯度下降过程实现点态收敛的深度神经网络。该DNN具有稀疏且特殊的形状，并包含特定的变量变换。