Creating high quality and realistic materials in computer graphics is a challenging and time-consuming task, which requires great expertise. In this paper, we present MatFuse, a novel unified approach that harnesses the generative power of diffusion models (DM) to simplify the creation of SVBRDF maps. Our DM-based pipeline integrates multiple sources of conditioning, such as color palettes, sketches, and pictures, enabling fine-grained control and flexibility in material synthesis. This design allows for the combination of diverse information sources (e.g., sketch + image embedding), enhancing creative possibilities in line with the principle of compositionality. We demonstrate the generative capabilities of the proposed method under various conditioning settings; on the SVBRDF estimation task, we show that our method yields performance comparable to state-of-the-art approaches, both qualitatively and quantitatively.

翻译：在计算机图形学中，创建高质量且逼真的材质是一项具有挑战性且耗时的工作，需要丰富的专业知识。本文提出MatFuse——一种新颖的统一方法，利用扩散模型的生成能力来简化SVBRDF贴图的创建。我们的基于扩散模型的流程整合了多种条件输入，如调色板、草图及图片，从而在材质合成中实现精细控制与灵活性。该设计允许结合多种信息源（例如草图+图像嵌入），遵循组合性原则以增强创作可能性。我们将在多种条件设置下展示所提方法的生成能力；在SVBRDF估计任务上，我们表明该方法无论是在定性还是定量评估中，均能取得与最先进方法相当的性能。