In this paper, we introduce a novel analysis of neural networks based on geometric (Clifford) algebra and convex optimization. We show that optimal weights of deep ReLU neural networks are given by the wedge product of training samples when trained with standard regularized loss. Furthermore, the training problem reduces to convex optimization over wedge product features, which encode the geometric structure of the training dataset. This structure is given in terms of signed volumes of triangles and parallelotopes generated by data vectors. The convex problem finds a small subset of samples via $\ell_1$ regularization to discover only relevant wedge product features. Our analysis provides a novel perspective on the inner workings of deep neural networks and sheds light on the role of the hidden layers.

翻译：本文提出了一种基于几何（克利福德）代数和凸优化的神经网络新分析法。我们证明，当使用标准正则化损失训练时，深度ReLU神经网络的最优权值由训练样本的楔积给出。此外，训练问题简化为对编码训练数据集几何结构的楔积特征进行凸优化。该结构以数据向量生成的三角形和平行多面体的有向体积形式表示。凸优化问题通过$\ell_1$正则化找到少量样本子集，以发现仅相关的楔积特征。我们的分析为深度神经网络的内部机制提供了新视角，并揭示了隐藏层的作用。