Neural functional networks (NFNs) have recently gained significant attention due to their diverse applications, ranging from predicting network generalization and network editing to classifying implicit neural representation. Previous NFN designs often depend on permutation symmetries in neural networks' weights, which traditionally arise from the unordered arrangement of neurons in hidden layers. However, these designs do not take into account the weight scaling symmetries of $\ReLU$ networks, and the weight sign flipping symmetries of $\sin$ or $\Tanh$ networks. In this paper, we extend the study of the group action on the network weights from the group of permutation matrices to the group of monomial matrices by incorporating scaling/sign-flipping symmetries. Particularly, we encode these scaling/sign-flipping symmetries by designing our corresponding equivariant and invariant layers. We name our new family of NFNs the Monomial Matrix Group Equivariant Neural Functional Networks (Monomial-NFN). Because of the expansion of the symmetries, Monomial-NFN has much fewer independent trainable parameters compared to the baseline NFNs in the literature, thus enhancing the model's efficiency. Moreover, for fully connected and convolutional neural networks, we theoretically prove that all groups that leave these networks invariant while acting on their weight spaces are some subgroups of the monomial matrix group. We provide empirical evidence to demonstrate the advantages of our model over existing baselines, achieving competitive performance and efficiency.

翻译：神经函数网络（NFN）近年来因其多样化的应用而受到广泛关注，其应用范围涵盖预测网络泛化能力、网络编辑以及隐式神经表示分类等。以往的NFN设计通常依赖于神经网络权重中的置换对称性，这种对称性传统上源于隐藏层神经元无序排列的特性。然而，这些设计未考虑$\ReLU$网络的权重缩放对称性，以及$\sin$或$\Tanh$网络的权重符号翻转对称性。本文通过引入缩放/符号翻转对称性，将网络权重上的群作用研究从置换矩阵群扩展至单项矩阵群。具体而言，我们通过设计相应的等变层与不变层来编码这些缩放/符号翻转对称性。我们将这一新型NFN家族命名为单项矩阵群等变神经函数网络（Monomial-NFN）。由于对称性的扩展，与现有文献中的基准NFN相比，Monomial-NFN具有更少的独立可训练参数，从而提升了模型效率。此外，针对全连接神经网络与卷积神经网络，我们从理论上证明了所有作用于权重空间且保持这些网络不变的群均为单项矩阵群的子群。我们通过实验证据展示了所提模型相较于现有基准方法的优势，在性能与效率方面均达到了具有竞争力的水平。