Diffusion models have been shown to be capable of generating high-quality images, suggesting that they could contain meaningful internal representations. Unfortunately, the feature maps that encode a diffusion model's internal information are spread not only over layers of the network, but also over diffusion timesteps, making it challenging to extract useful descriptors. We propose Diffusion Hyperfeatures, a framework for consolidating multi-scale and multi-timestep feature maps into per-pixel feature descriptors that can be used for downstream tasks. These descriptors can be extracted for both synthetic and real images using the generation and inversion processes. We evaluate the utility of our Diffusion Hyperfeatures on the task of semantic keypoint correspondence: our method achieves superior performance on the SPair-71k real image benchmark. We also demonstrate that our method is flexible and transferable: our feature aggregation network trained on the inversion features of real image pairs can be used on the generation features of synthetic image pairs with unseen objects and compositions. Our code is available at \url{https://diffusion-hyperfeatures.github.io}.

翻译：扩散模型已被证明能够生成高质量图像，表明它们可能包含有意义的内部表征。然而，编码扩散模型内部信息的特征图不仅分布在网络的各层，还分布在扩散时间步中，这使得提取有用的描述符变得具有挑战性。我们提出了扩散超特征（Diffusion Hyperfeatures）框架，该框架将多尺度、多时间步的特征图整合为逐像素的特征描述符，可用于下游任务。通过生成和反演过程，这些描述符可同时从合成图像与真实图像中提取。我们在语义关键点对应任务上评估了扩散超特征的实用性：我们的方法在SPair-71k真实图像基准上取得了优越的性能。我们还证明了该方法具有灵活性和可迁移性：基于真实图像对反演特征训练得到的特征聚合网络，可直接用于包含未见物体及其组合的合成图像对的生成特征。我们的代码已开源发布于 \url{https://diffusion-hyperfeatures.github.io}。