We introduce meta-learning algorithms that perform zero-shot weight-space adaptation of neural network models to unseen tasks. Our methods repurpose the popular generative image synthesis techniques of natural language guidance and diffusion models to generate neural network weights adapted for tasks. We first train an unconditional generative hypernetwork model to produce neural network weights; then we train a second "guidance" model that, given a natural language task description, traverses the hypernetwork latent space to find high-performance task-adapted weights in a zero-shot manner. We explore two alternative approaches for latent space guidance: "HyperCLIP"-based classifier guidance and a conditional Hypernetwork Latent Diffusion Model ("HyperLDM"), which we show to benefit from the classifier-free guidance technique common in image generation. Finally, we demonstrate that our approaches outperform existing multi-task and meta-learning methods in a series of zero-shot learning experiments on our Meta-VQA dataset.

翻译：我们提出了元学习算法，用于对神经网络模型进行零样本权重空间自适应，以应对未见过的任务。我们的方法借鉴了自然语言引导和扩散模型等流行的生成式图像合成技术，生成适应任务的神经网络权重。我们首先训练一个无条件生成的超网络模型，用于产生神经网络权重；随后训练第二个“引导”模型，该模型根据自然语言任务描述，在超网络潜在空间中进行遍历，以零样本方式找到高性能的任务自适应权重。我们探索了两种潜在空间引导的替代方法：基于“HyperCLIP”的分类器引导，以及条件性超网络潜在扩散模型（“HyperLDM”），并证明其受益于图像生成中常见的无分类器引导技术。最后，我们通过一系列在Meta-VQA数据集上的零样本学习实验，展示了我们的方法优于现有的多任务和元学习方法。