The growing proliferation of customized and pretrained generative models has made it infeasible for a user to be fully cognizant of every model in existence. To address this need, we introduce the task of content-based model search: given a query and a large set of generative models, finding the models that best match the query. As each generative model produces a distribution of images, we formulate the search task as an optimization problem to select the model with the highest probability of generating similar content as the query. We introduce a formulation to approximate this probability given the query from different modalities, e.g., image, sketch, and text. Furthermore, we propose a contrastive learning framework for model retrieval, which learns to adapt features for various query modalities. We demonstrate that our method outperforms several baselines on Generative Model Zoo, a new benchmark we create for the model retrieval task.

翻译：定制化和预训练生成模型的日益普及使得用户难以全面了解已存在的每一个模型。为解决这一需求，我们引入了基于内容的模型搜索任务：给定一个查询和大量生成模型，找到与查询最匹配的模型。由于每个生成模型都会生成一个图像分布，我们将搜索任务形式化为一个优化问题，以选择生成与查询相似内容概率最高的模型。我们提出了一种公式，用于根据来自不同模态（如图像、草图、文本）的查询来近似计算此概率。此外，我们提出了一种用于模型检索的对比学习框架，该框架能够学习为各种查询模态调整特征。我们在生成模型动物园（Generative Model Zoo）这一新创建的模型检索基准上，证明了我们的方法优于多个基线方法。