We propose a method to control material attributes of objects like roughness, metallic, albedo, and transparency in real images. Our method capitalizes on the generative prior of text-to-image models known for photorealism, employing a scalar value and instructions to alter low-level material properties. Addressing the lack of datasets with controlled material attributes, we generated an object-centric synthetic dataset with physically-based materials. Fine-tuning a modified pre-trained text-to-image model on this synthetic dataset enables us to edit material properties in real-world images while preserving all other attributes. We show the potential application of our model to material edited NeRFs.

翻译：我们提出一种方法，用于控制真实图像中物体的材质属性，如粗糙度、金属度、反照率和透明度。该方法利用以照片级真实感著称的文本到图像模型的生成先验，通过标量值和指令来调整底层材质属性。针对缺乏具有可控材质属性的数据集的问题，我们生成了一个基于物理材质的物体中心合成数据集。在此合成数据集上微调改进后的预训练文本到图像模型，使我们能够在保留所有其他属性的同时编辑真实世界图像中的材质属性。我们还展示了该模型在材质编辑神经辐射场（NeRF）中的潜在应用。