In this paper, we introduce a novel analysis of neural networks based on geometric (Clifford) algebra and convex optimization. We show that optimal weights of deep ReLU neural networks are given by the wedge product of training samples when trained with standard regularized loss. Furthermore, the training problem reduces to convex optimization over wedge product features, which encode the geometric structure of the training dataset. This structure is given in terms of signed volumes of triangles and parallelotopes generated by data vectors. The convex problem finds a small subset of samples via $\ell_1$ regularization to discover only relevant wedge product features. Our analysis provides a novel perspective on the inner workings of deep neural networks and sheds light on the role of the hidden layers.

翻译：本文基于几何（克利福德）代数和凸优化提出了一种新颖的神经网络分析方法。我们证明，在使用标准正则化损失函数训练时，深度ReLU神经网络的最优权重由训练样本的楔积给出。进一步，训练问题可简化为对编码训练数据集几何结构的楔积特征进行凸优化，该结构由数据向量生成的三角形和平行多面体的有符号体积表示。凸优化通过ℓ1正则化寻找少量样本子集，以仅发现相关的楔积特征。我们的分析为深度神经网络的内部工作机制提供了新视角，并揭示了隐藏层的作用。