Although deep learning (DL) has led to several breakthroughs in many disciplines as diverse as chemistry, computer science, electrical engineering, mathematics, medicine, neuroscience, and physics, a comprehensive understanding of why and how DL is empirically successful remains fundamentally elusive. To attack this fundamental problem and unravel the mysteries behind DL's empirical successes, significant innovations toward a unified theory of DL have been made. These innovations encompass nearly fundamental advances in optimization, generalization, and approximation. Despite these advances, however, no work to date has offered a way to quantify the testing performance of a DL-based algorithm employed to solve a pattern classification problem. To overcome this fundamental challenge in part, this paper exposes the fundamental testing performance limits of DL-based binary classifiers trained with hinge loss. For binary classifiers that are based on deep rectified linear unit (ReLU) feedforward neural networks (FNNs) and ones that are based on deep FNNs with ReLU and Tanh activation, we derive their respective novel asymptotic testing performance limits. The derived testing performance limits are validated by extensive computer experiments.

翻译：尽管深度学习已在化学、计算机科学、电气工程、数学、医学、神经科学和物理学等诸多学科领域取得突破性进展，但对其经验成功背后"为何"与"如何"的根本性理解仍基本缺失。为攻克这一根本性问题并揭示深度学习经验成功之谜，研究者已在统一深度学习理论方面取得重大创新，这些创新基本涵盖了优化、泛化和近似三大领域的理论进展。然而，迄今尚无工作提供量化基于深度学习的模式分类算法测试性能的方法。为部分攻克这一根本性挑战，本文揭示了基于铰链损失训练的深度学习二分类器的基本测试性能极限。针对基于深度修正线性单元前馈神经网络与基于含修正线性单元及双曲正切激活函数深度前馈神经网络的二分类器，我们分别推导了其新型渐近测试性能极限。所推导的测试性能极限通过大量计算机实验得到验证。