We present a novel application of category theory for deep learning. We show how category theory can be used to understand and work with the linear layer functions of group equivariant neural networks whose layers are some tensor power space of $\mathbb{R}^{n}$ for the groups $S_n$, $O(n)$, $Sp(n)$, and $SO(n)$. By using category theoretic constructions, we build a richer structure that is not seen in the original formulation of these neural networks, leading to new insights. In particular, we outline the development of an algorithm for quickly computing the result of a vector that is passed through an equivariant, linear layer for each group in question. The success of our approach suggests that category theory could be beneficial for other areas of deep learning.
翻译:我们提出了一种范畴论在深度学习中的新颖应用。我们展示了如何利用范畴论来理解和处理群等变神经网络的线性层函数,这些网络的层是$\mathbb{R}^{n}$的张量幂空间,作用于群$S_n$、$O(n)$、$Sp(n)$和$SO(n)$。通过使用范畴论构造,我们构建了一个比这些神经网络原始表述更丰富的结构,从而引发新的见解。特别地,我们概述了一种算法的开发过程,该算法能够快速计算向量通过每个相关群的等变线性层后的结果。我们方法的成功表明,范畴论可能有利于深度学习的其他领域。