This paper considers deep neural networks for learning weakly dependent processes in a general framework that includes, for instance, regression estimation, time series prediction, time series classification. The $\psi$-weak dependence structure considered is quite large and covers other conditions such as mixing, association,$\ldots$ Firstly, the approximation of smooth functions by deep neural networks with a broad class of activation functions is considered. We derive the required depth, width and sparsity of a deep neural network to approximate any H\"{o}lder smooth function, defined on any compact set $\mx$. Secondly, we establish a bound of the excess risk for the learning of weakly dependent observations by deep neural networks. When the target function is sufficiently smooth, this bound is close to the usual $\mathcal{O}(n^{-1/2})$.
翻译:本文在通用框架下研究深度神经网络对弱相依过程的学习问题,该框架涵盖回归估计、时间序列预测、时间序列分类等应用场景。所考虑的$\psi$-弱相依结构具有广泛适用性,可涵盖混合性、关联性等其他条件。首先,本文探讨了具有宽泛激活函数类的深度神经网络对光滑函数的逼近能力,推导了在紧集$\mx$上逼近任意Hölder光滑函数时所需的网络深度、宽度及稀疏性条件。其次,针对深度神经网络学习弱相依观测数据的问题,建立了超风险界。当目标函数足够光滑时,该界趋近于常规的$\mathcal{O}(n^{-1/2})$阶。