In this work, we explore intersections between sparse coding and deep learning to enhance our understanding of feature extraction capabilities in advanced neural network architectures. We begin by introducing a novel class of Deep Sparse Coding (DSC) models and establish thorough theoretical analysis of their uniqueness and stability properties. By applying iterative algorithms to these DSC models, we derive convergence rates for convolutional neural networks (CNNs) in their ability to extract sparse features. This provides a strong theoretical foundation for the use of CNNs in sparse feature learning tasks. We additionally extend the convergence analysis to more general neural network architectures, including those with diverse activation functions, as well as self-attention and transformer-based models. This broadens the applicability of our findings to a wide range of deep learning methods for deep sparse feature extraction. Inspired by the strong connection between sparse coding and CNNs, we also explore training strategies to encourage neural networks to learn more sparse features. Through numerical experiments, we demonstrate the effectiveness of these approaches, providing valuable insights for the design of efficient and interpretable deep learning models.

翻译：本文探索了稀疏编码与深度学习之间的交叉领域，以深化对先进神经网络架构特征提取能力的理解。我们首先提出一类新型深度稀疏编码模型，并对其唯一性和稳定性进行了完整的理论分析。通过对这些DSC模型应用迭代算法，我们推导出卷积神经网络提取稀疏特征的收敛速率，为CNN在稀疏特征学习任务中的应用奠定了坚实的理论基础。此外，我们将收敛性分析扩展到更广泛的神经网络架构，包括具有多样化激活函数的网络，以及基于自注意力机制和Transformer的模型，从而将研究结论的适用范围拓展至多种深度稀疏特征提取方法。受稀疏编码与CNN之间紧密联系的启发，我们还探索了促使神经网络学习更稀疏特征的训练策略。通过数值实验，我们验证了这些方法的有效性，为设计高效且可解释的深度学习模型提供了重要见解。