Work on fast weight programmers has demonstrated the effectiveness of key/value outer product-based learning rules for sequentially generating a weight matrix (WM) of a neural net (NN) by another NN or itself. However, the weight generation steps are typically not visually interpretable by humans, because the contents stored in the WM of an NN are not. Here we apply the same principle to generate natural images. The resulting fast weight painters (FPAs) learn to execute sequences of delta learning rules to sequentially generate images as sums of outer products of self-invented keys and values, one rank at a time, as if each image was a WM of an NN. We train our FPAs in the generative adversarial networks framework, and evaluate on various image datasets. We show how these generic learning rules can generate images with respectable visual quality without any explicit inductive bias for images. While the performance largely lags behind the one of specialised state-of-the-art image generators, our approach allows for visualising how synaptic learning rules iteratively produce complex connection patterns, yielding human-interpretable meaningful images. Finally, we also show that an additional convolutional U-Net (now popular in diffusion models) at the output of an FPA can learn one-step "denoising" of FPA-generated images to enhance their quality. Our code is public.

翻译：快速权重编程的研究已经证明了基于键值外积的学习规则在通过另一个神经网络或自身顺序生成神经网络权重矩阵方面的有效性。然而，由于神经网络权重矩阵中存储的内容通常无法被人眼直观解读，权重生成步骤往往缺乏可视化的可解释性。本文将该原理应用于自然图像生成领域。由此产生的快速权重画师（FPA）学会执行一系列增量学习规则，以每次一秩的方式，通过自创键与值的外积之和来顺序生成图像，如同每幅图像都是一个神经网络的权重矩阵。我们在生成对抗网络框架下训练FPA，并在多种图像数据集上评估其性能。研究表明，这些通用学习规则无需任何针对图像的显式归纳偏置即可生成具有可观视觉质量的图像。尽管其性能在很大程度上落后于专门的最先进图像生成器，但我们的方法能够展示突触学习规则如何迭代产生复杂连接模式，从而生成人类可理解的有意义图像。最后，我们还证明，在FPA输出端额外添加卷积U-Net（当前在扩散模型中广泛使用）可以学习单步"去噪"FPA生成的图像，从而提升其质量。我们的代码已公开。