Creating high-quality materials in computer graphics is a challenging and time-consuming task, which requires great expertise. To simplify this process, we introduce MatFuse, a unified approach that harnesses the generative power of diffusion models for creation and editing of 3D materials. Our method integrates multiple sources of conditioning, including color palettes, sketches, text, and pictures, enhancing creative possibilities and granting fine-grained control over material synthesis. Additionally, MatFuse enables map-level material editing capabilities through latent manipulation by means of a multi-encoder compression model which learns a disentangled latent representation for each map. We demonstrate the effectiveness of MatFuse under multiple conditioning settings and explore the potential of material editing. Finally, we assess the quality of the generated materials both quantitatively in terms of CLIP-IQA and FID scores and qualitatively by conducting a user study. Source code for training MatFuse and supplemental materials are publicly available at https://gvecchio.com/matfuse.

翻译：计算机图形学中创建高质量材质是一项具有挑战性且耗时的任务，需要丰富的专业知识。为简化这一流程，我们提出MatFuse——一种利用扩散模型生成能力进行三维材质创建与编辑的统一方法。该方法集成了多种条件输入，包括调色板、草图、文本和图片，从而增强创作可能性并实现材质合成的精细控制。此外，MatFuse通过多编码器压缩模型对潜在空间进行操作，实现了贴图级别的材质编辑能力——该模型为每张贴图学习解耦的潜在表征。我们验证了MatFuse在多种条件设置下的有效性，并探索了材质编辑的潜力。最后，我们分别通过CLIP-IQA和FID评分进行定量评估，并通过用户研究进行定性分析来衡量生成材质的质量。训练MatFuse的源代码及补充材料已公开于 https://gvecchio.com/matfuse。