In this paper, we introduce a novel analysis of neural networks based on geometric (Clifford) algebra and convex optimization. We show that optimal weights of deep ReLU neural networks are given by the wedge product of training samples when trained with standard regularized loss. Furthermore, the training problem reduces to convex optimization over wedge product features, which encode the geometric structure of the training dataset. This structure is given in terms of signed volumes of triangles and parallelotopes generated by data vectors. The convex problem finds a small subset of samples via $\ell_1$ regularization to discover only relevant wedge product features. Our analysis provides a novel perspective on the inner workings of deep neural networks and sheds light on the role of the hidden layers.

翻译：本文提出了一种基于几何（克利福德）代数与凸优化的神经网络新分析方法。我们证明，在使用标准正则化损失训练时，深度ReLU神经网络的最优权重由训练样本的楔积给出。进一步地，训练问题可归结为对编码训练数据集几何结构的楔积特征进行凸优化。该结构以数据向量生成的三角形和平行多面体的有向体积形式呈现。通过ℓ1正则化，该凸问题能够筛选出少量样本，仅发现相关的楔积特征。我们的分析为深度神经网络的内部运作机制提供了全新视角，并揭示了隐藏层的作用机理。