Coordinate-based Multilayer Perceptron (MLP) networks, despite being capable of learning neural implicit representations, are not performant for internal image synthesis applications. Convolutional Neural Networks (CNNs) are typically used instead for a variety of internal generative tasks, at the cost of a larger model. We propose Neural Knitwork, an architecture for neural implicit representation learning of natural images that achieves image synthesis by optimizing the distribution of image patches in an adversarial manner and by enforcing consistency between the patch predictions. To the best of our knowledge, this is the first implementation of a coordinate-based MLP tailored for synthesis tasks such as image inpainting, super-resolution, and denoising. We demonstrate the utility of the proposed technique by training on these three tasks. The results show that modeling natural images using patches, rather than pixels, produces results of higher fidelity. The resulting model requires 80% fewer parameters than alternative CNN-based solutions while achieving comparable performance and training time.

翻译：基于坐标的多层感知器（MLP）网络尽管能够学习神经隐式表示，但在内部图像合成应用中表现不佳。卷积神经网络（CNN）通常用于各种内部生成任务，但代价是模型规模更大。我们提出神经编织网络（Neural Knitwork），一种用于自然图像神经隐式表示学习的架构，该架构通过对抗性方式优化图像补丁分布并强制补丁预测间的一致性来实现图像合成。据我们所知，这是首个专为图像修复、超分辨率和去噪等合成任务定制的基于坐标的MLP实现。我们通过在三种任务上的训练验证了所提技术的实用性。结果表明，使用补丁而非像素对自然图像建模可产生更高保真度的结果。与基于CNN的替代方案相比，所提模型参数量减少80%，同时达到相当的性能和训练时间。