Automatic 3D content creation has achieved rapid progress recently due to the availability of pre-trained, large language models and image diffusion models, forming the emerging topic of text-to-3D content creation. Existing text-to-3D methods commonly use implicit scene representations, which couple the geometry and appearance via volume rendering and are suboptimal in terms of recovering finer geometries and achieving photorealistic rendering; consequently, they are less effective for generating high-quality 3D assets. In this work, we propose a new method of Fantasia3D for high-quality text-to-3D content creation. Key to Fantasia3D is the disentangled modeling and learning of geometry and appearance. For geometry learning, we rely on a hybrid scene representation, and propose to encode surface normal extracted from the representation as the input of the image diffusion model. For appearance modeling, we introduce the spatially varying bidirectional reflectance distribution function (BRDF) into the text-to-3D task, and learn the surface material for photorealistic rendering of the generated surface. Our disentangled framework is more compatible with popular graphics engines, supporting relighting, editing, and physical simulation of the generated 3D assets. We conduct thorough experiments that show the advantages of our method over existing ones under different text-to-3D task settings. Project page and source codes: https://fantasia3d.github.io/.

翻译：近年来，由于预训练大语言模型和图像扩散模型的出现，自动3D内容创建取得了快速进展，形成了新兴的文本到3D内容创建课题。现有的文本到3D方法通常使用隐式场景表示，通过体渲染将几何与外观耦合，在恢复精细几何和实现逼真渲染方面存在不足，因此难以生成高质量3D资产。本文提出了一种名为Fantasia3D的新方法，用于高质量文本到3D内容创建。Fantasia3D的关键在于对几何与外观进行解耦建模与学习。在几何学习方面，我们采用混合场景表示，并提议将从该表示中提取的表面法线编码为图像扩散模型的输入。在外观建模方面，我们将空间变化的双向反射分布函数（BRDF）引入文本到3D任务，并学习表面材质以实现生成表面的逼真渲染。我们的解耦框架与主流图形引擎更兼容，支持生成的3D资产的重新照明、编辑和物理模拟。我们进行了充分的实验，展示了我们的方法在不同文本到3D任务设置下相较于现有方法的优势。项目页面和源代码：https://fantasia3d.github.io/。