Sparse deep learning has become a popular technique for improving the performance of deep neural networks in areas such as uncertainty quantification, variable selection, and large-scale network compression. However, most existing research has focused on problems where the observations are independent and identically distributed (i.i.d.), and there has been little work on the problems where the observations are dependent, such as time series data and sequential data in natural language processing. This paper aims to address this gap by studying the theory for sparse deep learning with dependent data. We show that sparse recurrent neural networks (RNNs) can be consistently estimated, and their predictions are asymptotically normally distributed under appropriate assumptions, enabling the prediction uncertainty to be correctly quantified. Our numerical results show that sparse deep learning outperforms state-of-the-art methods, such as conformal predictions, in prediction uncertainty quantification for time series data. Furthermore, our results indicate that the proposed method can consistently identify the autoregressive order for time series data and outperform existing methods in large-scale model compression. Our proposed method has important practical implications in fields such as finance, healthcare, and energy, where both accurate point estimates and prediction uncertainty quantification are of concern.

翻译：稀疏深度学习已成为提升深度神经网络在不确定性量化、变量选择及大规模网络压缩等任务中性能的流行技术。然而，现有研究多聚焦于观测数据独立同分布（i.i.d.）的问题，而针对观测数据存在依赖关系（如时间序列数据及自然语言处理中的序列数据）的研究尚属少见。本文旨在填补这一空白，系统研究依赖数据场景下的稀疏深度学习理论。我们证明，在适当假设下，稀疏循环神经网络（RNNs）可被一致估计，且其预测值渐近服从正态分布，从而能正确量化预测不确定性。数值结果表明，在时间序列数据的预测不确定性量化任务中，稀疏深度学习优于共形预测等前沿方法。此外，研究结果显示，提出的方法能够一致识别时间序列数据的自回归阶数，并在大规模模型压缩中超越现有方法。本方法在金融、医疗及能源等既需精确点估计又需预测不确定性量化的领域具有重要实践意义。