Neural networks that process the parameters of other neural networks find applications in domains as diverse as classifying implicit neural representations, generating neural network weights, and predicting generalization errors. However, existing approaches either overlook the inherent permutation symmetry in the neural network or rely on intricate weight-sharing patterns to achieve equivariance, while ignoring the impact of the network architecture itself. In this work, we propose to represent neural networks as computational graphs of parameters, which allows us to harness powerful graph neural networks and transformers that preserve permutation symmetry. Consequently, our approach enables a single model to encode neural computational graphs with diverse architectures. We showcase the effectiveness of our method on a wide range of tasks, including classification and editing of implicit neural representations, predicting generalization performance, and learning to optimize, while consistently outperforming state-of-the-art methods. The source code is open-sourced at https://github.com/mkofinas/neural-graphs.

翻译：处理其他神经网络参数的神经网络在隐式神经表征分类、神经网络权重生成及泛化误差预测等多样化领域具有广泛应用。然而，现有方法要么忽视神经网络中固有的置换对称性，要么依赖复杂权重共享模式实现等变性，同时忽略网络架构本身的影响。本研究提出将神经网络表示为参数计算图，该方法能够利用保留置换对称性的强大图神经网络与Transformer模型。由此，单一模型即可编码具有多样架构的神经计算图。我们在隐式神经表征分类与编辑、泛化性能预测及优化学习等多项任务中验证了该方法的效果，始终优于现有最优方法。源代码已开源至https://github.com/mkofinas/neural-graphs。