In this paper, we develop a general theory for adaptive nonparametric estimation of the mean function of a non-stationary and nonlinear time series model using deep neural networks (DNNs). We first consider two types of DNN estimators, non-penalized and sparse-penalized DNN estimators, and establish their generalization error bounds for general non-stationary time series. We then derive minimax lower bounds for estimating mean functions belonging to a wide class of nonlinear autoregressive (AR) models that include nonlinear generalized additive AR, single index, and threshold AR models. Building upon the results, we show that the sparse-penalized DNN estimator is adaptive and attains the minimax optimal rates up to a poly-logarithmic factor for many nonlinear AR models. Through numerical simulations, we demonstrate the usefulness of the DNN methods for estimating nonlinear AR models with intrinsic low-dimensional structures and discontinuous or rough mean functions, which is consistent with our theory.

翻译：本文针对非平稳非线性时间序列模型，发展了一套使用深度神经网络（DNN）对均值函数进行自适应非参数估计的一般理论。我们首先考虑两类DNN估计量——无惩罚项和稀疏惩罚项的DNN估计量，并建立了它们在一般非平稳时间序列下的泛化误差界。随后，我们推导了估计属于一大类非线性自回归（AR）模型（包括非线性广义可加AR、单指标AR和门限AR模型）的均值函数的极小化最优下界。基于这些结果，我们证明了稀疏惩罚项DNN估计量具有自适应性，并且对于许多非线性AR模型，其收敛速度可达到仅差多项式对数因子的极小化最优速度。通过数值模拟，我们展示了DNN方法在估计具有内在低维结构以及不连续或粗糙均值函数的非线性AR模型方面的有效性，这与我们的理论结果一致。