Automatic 3D content creation has achieved rapid progress recently due to the availability of pre-trained, large language models and image diffusion models, forming the emerging topic of text-to-3D content creation. Existing text-to-3D methods commonly use implicit scene representations, which couple the geometry and appearance via volume rendering and are suboptimal in terms of recovering finer geometries and achieving photorealistic rendering; consequently, they are less effective for generating high-quality 3D assets. In this work, we propose a new method of Fantasia3D for high-quality text-to-3D content creation. Key to Fantasia3D is the disentangled modeling and learning of geometry and appearance. For geometry learning, we rely on a hybrid scene representation, and propose to encode surface normal extracted from the representation as the input of the image diffusion model. For appearance modeling, we introduce the spatially varying bidirectional reflectance distribution function (BRDF) into the text-to-3D task, and learn the surface material for photorealistic rendering of the generated surface. Our disentangled framework is more compatible with popular graphics engines, supporting relighting, editing, and physical simulation of the generated 3D assets. We conduct thorough experiments that show the advantages of our method over existing ones under different text-to-3D task settings. Project page and source codes: https://fantasia3d.github.io/.

翻译：最近，由于预训练的大语言模型和图像扩散模型的可用性，自动3D内容创建取得了快速进展，形成了新兴的文本到3D内容生成课题。现有的文本到3D方法通常使用隐式场景表示，通过体渲染耦合几何与外观，在恢复更精细几何和实现逼真渲染方面并非最优；因此，它们在生成高质量3D资产方面效果较差。在这项工作中，我们提出了一种名为Fantasia3D的新方法，用于高质量文本到3D内容生成。Fantasia3D的关键在于几何与外观的解耦建模和学习。对于几何学习，我们依赖一种混合场景表示，并提议将从该表示中提取的表面法线编码为图像扩散模型的输入。对于外观建模，我们将空间变化的双向反射分布函数（BRDF）引入文本到3D任务，并学习表面材料以实现生成表面的逼真渲染。我们的解耦框架与流行的图形引擎更兼容，支持生成3D资产的重新光照、编辑和物理模拟。我们进行了彻底的实验，展示了在不同文本到3D任务设置下，我们的方法相较于现有方法的优势。项目页面和源代码：https://fantasia3d.github.io/。