In the pursuit of efficient automated content creation, procedural generation, leveraging modifiable parameters and rule-based systems, emerges as a promising approach. Nonetheless, it could be a demanding endeavor, given its intricate nature necessitating a deep understanding of rules, algorithms, and parameters. To reduce workload, we introduce 3D-GPT, a framework utilizing large language models~(LLMs) for instruction-driven 3D modeling. 3D-GPT positions LLMs as proficient problem solvers, dissecting the procedural 3D modeling tasks into accessible segments and appointing the apt agent for each task. 3D-GPT integrates three core agents: the task dispatch agent, the conceptualization agent, and the modeling agent. They collaboratively achieve two objectives. First, it enhances concise initial scene descriptions, evolving them into detailed forms while dynamically adapting the text based on subsequent instructions. Second, it integrates procedural generation, extracting parameter values from enriched text to effortlessly interface with 3D software for asset creation. Our empirical investigations confirm that 3D-GPT not only interprets and executes instructions, delivering reliable results but also collaborates effectively with human designers. Furthermore, it seamlessly integrates with Blender, unlocking expanded manipulation possibilities. Our work highlights the potential of LLMs in 3D modeling, offering a basic framework for future advancements in scene generation and animation.

翻译：在追求高效自动化内容创作的过程中，程序化生成凭借其可修改的参数和基于规则的系统，成为一种前景广阔的方法。然而，由于其复杂性需要对规则、算法和参数有深入理解，这可能是一项要求苛刻的工作。为了减少工作量，我们提出了3D-GPT，这是一个利用大语言模型（LLMs）进行指令驱动三维建模的框架。3D-GPT将LLMs定位为熟练的问题解决者，将程序化三维建模任务分解为易于处理的片段，并为每个任务分配合适的智能体。3D-GPT集成了三个核心智能体：任务调度智能体、概念化智能体和建模智能体。它们协同工作以实现两个目标。首先，它增强简洁的初始场景描述，将其演化为详细形式，同时根据后续指令动态调整文本。其次，它整合程序化生成，从丰富的文本中提取参数值，以便轻松与三维软件接口进行资产创建。我们的实证研究证实，3D-GPT不仅能解释和执行指令，提供可靠的结果，还能与人类设计师有效协作。此外，它能与Blender无缝集成，解锁更多操作可能性。我们的工作凸显了LLMs在三维建模中的潜力，为未来场景生成和动画的进展提供了一个基础框架。