Neural Representations have recently been shown to effectively reconstruct a wide range of signals from 3D meshes and shapes to images and videos. We show that, when adapted correctly, neural representations can be used to directly represent the weights of a pre-trained convolutional neural network, resulting in a Neural Representation for Neural Networks (NeRN). Inspired by coordinate inputs of previous neural representation methods, we assign a coordinate to each convolutional kernel in our network based on its position in the architecture, and optimize a predictor network to map coordinates to their corresponding weights. Similarly to the spatial smoothness of visual scenes, we show that incorporating a smoothness constraint over the original network's weights aids NeRN towards a better reconstruction. In addition, since slight perturbations in pre-trained model weights can result in a considerable accuracy loss, we employ techniques from the field of knowledge distillation to stabilize the learning process. We demonstrate the effectiveness of NeRN in reconstructing widely used architectures on CIFAR-10, CIFAR-100, and ImageNet. Finally, we present two applications using NeRN, demonstrating the capabilities of the learned representations.

翻译：近期研究表明，神经表示能够有效重建从三维网格、形状到图像与视频的广泛信号。本文证明，通过适当调整，神经表示可直接用于表示预训练卷积神经网络的权重，由此提出面向神经网络的神经由表示方法（NeRN）。受先前神经表示方法中坐标输入的启发，我们根据卷积核在架构中的位置为其分配坐标，并优化一个预测器网络将坐标映射至对应权重。与视觉场景的空间平滑性类似，我们发现对原始网络权重施加平滑约束有助于NeRN实现更优重建。此外，由于预训练模型权重的微小扰动可能导致显著精度损失，我们采用知识蒸馏领域的技术来稳定学习过程。我们通过CIFAR-10、CIFAR-100和ImageNet数据集上的实验验证了NeRN对主流架构的重建有效性。最后，我们展示了NeRN的两项应用实例，以论证所学表示的能力。