Neural networks that satisfy invariance with respect to input permutations have been widely studied in machine learning literature. However, in many applications, only a subset of all input permutations is of interest. For heterogeneous graph data, one can focus on permutations that preserve node types. We fully characterize linear layers invariant to such permutations. We verify experimentally that implementing these layers in graph neural network architectures allows learning important node interactions more effectively than existing techniques. We show that the dimension of space of these layers is given by a generalization of Bell numbers, extending the work (Maron et al., 2019). We further narrow the invariant network design space by addressing a question about the sizes of tensor layers necessary for function approximation on graph data. Our findings suggest that function approximation on a graph with $n$ nodes can be done with tensors of sizes $\leq n$, which is tighter than the best-known bound $\leq n(n-1)/2$. For $d \times d$ image data with translation symmetry, our methods give a tight upper bound $2d - 1$ (instead of $d^{4}$) on sizes of invariant tensor generators via a surprising connection to Davenport constants.

翻译：满足输入置换不变性的神经网络已在机器学习文献中得到广泛研究。然而在许多应用中，仅需关注全部输入置换中的特定子集。对于异构图数据，我们可聚焦于保持节点类型不变的置换。本文完整刻画了对此类置换具有不变性的线性层。实验验证表明，在图表征学习架构中应用这些层，比现有技术能更有效地学习重要节点交互。我们证明此类线性层的空间维度可由贝尔数的广义形式给出，从而扩展了Maron等人(2019)的研究工作。进一步地，通过探讨图数据函数逼近所需的张量层维度问题，我们收窄了不变网络的设计空间。研究发现，包含$n$个节点的图可通过尺寸$\leq n$的张量实现函数逼近，这比已知最优界$\leq n(n-1)/2$更为紧致。针对具有平移对称性的$d \times d$图像数据，我们的方法通过达文波特常数的惊人关联，给出了不变张量生成器尺寸的紧致上界$2d - 1$（而非$d^{4}$）。