We propose ZeST, a method for zero-shot material transfer to an object in the input image given a material exemplar image. ZeST leverages existing diffusion adapters to extract implicit material representation from the exemplar image. This representation is used to transfer the material using pre-trained inpainting diffusion model on the object in the input image using depth estimates as geometry cue and grayscale object shading as illumination cues. The method works on real images without any training resulting a zero-shot approach. Both qualitative and quantitative results on real and synthetic datasets demonstrate that ZeST outputs photorealistic images with transferred materials. We also show the application of ZeST to perform multiple edits and robust material assignment under different illuminations. Project Page: https://ttchengab.github.io/zest

翻译：我们提出ZeST方法，用于在给定材质示例图像的情况下，对输入图像中的物体进行零样本材质迁移。ZeST利用现有扩散适配器从示例图像中提取隐式材质表征，并借助预训练修复扩散模型，以深度估计作为几何线索、灰度物体着色作为光照线索，将该表征迁移至输入图像中的物体上。该方法无需任何训练即可应用于真实图像，实现零样本处理。在真实与合成数据集上的定性与定量结果表明，ZeST能够生成具有迁移材质的光真实感图像。我们还展示了ZeST在多重编辑以及不同光照下稳健材质分配中的应用。项目页面：https://ttchengab.github.io/zest