In this paper, we develop a general theory for adaptive nonparametric estimation of the mean function of a non-stationary and nonlinear time series model using deep neural networks (DNNs). We first consider two types of DNN estimators, non-penalized and sparse-penalized DNN estimators, and establish their generalization error bounds for general non-stationary time series. We then derive minimax lower bounds for estimating mean functions belonging to a wide class of nonlinear autoregressive (AR) models that include nonlinear generalized additive AR, single index, and threshold AR models. Building upon the results, we show that the sparse-penalized DNN estimator is adaptive and attains the minimax optimal rates up to a poly-logarithmic factor for many nonlinear AR models. Through numerical simulations, we demonstrate the usefulness of the DNN methods for estimating nonlinear AR models with intrinsic low-dimensional structures and discontinuous or rough mean functions, which is consistent with our theory.

翻译：本文利用深度神经网络（DNNs）发展了一套针对非平稳非线性时间序列模型均值函数的自适应非参数估计通用理论。首先考虑两种DNN估计器——无惩罚和稀疏惩罚DNN估计器，并建立它们在一般非平稳时间序列上的泛化误差界。随后，针对一类包含非线性广义加性自回归(AR)、单指标及阈值AR模型的广义非线性自回归模型，推导了其均值函数估计的极小化最优下界。基于上述结果，我们证明稀疏惩罚DNN估计器具有自适应性，能够对多种非线性AR模型达到（至多相差一个多项式对数因子）极小化最优收敛速率。通过数值模拟，我们验证了DNN方法在估计具有内在低维结构及非连续或粗糙均值函数的非线性AR模型中的有效性，这与我们的理论结果相符。