Despite their simple intuition, convolutions are more tedious to analyze than dense layers, which complicates the generalization of theoretical and algorithmic ideas. We provide a new perspective onto convolutions through tensor networks (TNs) which allow reasoning about the underlying tensor multiplications by drawing diagrams, and manipulating them to perform function transformations, sub-tensor access, and fusion. We demonstrate this expressive power by deriving the diagrams of various autodiff operations and popular approximations of second-order information with full hyper-parameter support, batching, channel groups, and generalization to arbitrary convolution dimensions. Further, we provide convolution-specific transformations based on the connectivity pattern which allow to re-wire and simplify diagrams before evaluation. Finally, we probe computational performance, relying on established machinery for efficient TN contraction. Our TN implementation speeds up a recently-proposed KFAC variant up to 4.5x and enables new hardware-efficient tensor dropout for approximate backpropagation.

翻译：尽管卷积的直觉直观，但其分析比全连接层更为繁琐，这限制了许多理论和算法思想的推广。我们通过张量网络（Tensor Networks, TNs）为卷积提供了全新视角——借助图结构推理底层张量乘法，并通过图操作实现函数变换、子张量访问与融合。我们通过推导各类自动微分操作及二阶信息近似方法的图表示来展示这种表达能力，这些方法支持完整超参数、批处理、通道分组以及任意维度卷积的泛化。进一步地，我们基于卷积特有的连接模式提出变换规则，可在计算前重构并简化图结构。最后，我们利用成熟的张量网络压缩计算框架评估性能：基于张量网络的实现使近期提出的KFAC变体加速达4.5倍，并实现了面向近似反向传播的新型硬件高效张量丢弃机制。