Deep neural networks (DNNs), particularly those using Rectified Linear Unit (ReLU) activation functions, have achieved remarkable success across diverse machine learning tasks, including image recognition, audio processing, and language modeling. Despite this success, the non-convex nature of DNN loss functions complicates optimization and limits theoretical understanding. In this paper, we highlight how recently developed convex equivalences of ReLU NNs and their connections to sparse signal processing models can address the challenges of training and understanding NNs. Recent research has uncovered several hidden convexities in the loss landscapes of certain NN architectures, notably two-layer ReLU networks and other deeper or varied architectures. This paper seeks to provide an accessible and educational overview that bridges recent advances in the mathematics of deep learning with traditional signal processing, encouraging broader signal processing applications.

翻译：深度神经网络（DNNs），特别是采用修正线性单元（ReLU）激活函数的网络，在图像识别、音频处理和语言建模等各类机器学习任务中取得了显著成功。然而，DNN损失函数的非凸性使得优化过程复杂化，并限制了理论理解。本文重点阐述近期发展的ReLU神经网络凸等价性及其与稀疏信号处理模型的联系，如何应对神经网络的训练与理解挑战。最新研究揭示了特定神经网络架构（尤其双层ReLU网络及其他深层或异构架构）损失函数景观中存在的多种隐藏凸性。本文旨在提供易于理解且兼具教育意义的概述，搭建深度学习数学与经典信号处理之间的桥梁，从而推动信号处理领域的更广泛应用。