In the pursuit of efficient automated content creation, procedural generation, leveraging modifiable parameters and rule-based systems, emerges as a promising approach. Nonetheless, it could be a demanding endeavor, given its intricate nature necessitating a deep understanding of rules, algorithms, and parameters. To reduce workload, we introduce 3D-GPT, a framework utilizing large language models~(LLMs) for instruction-driven 3D modeling. 3D-GPT positions LLMs as proficient problem solvers, dissecting the procedural 3D modeling tasks into accessible segments and appointing the apt agent for each task. 3D-GPT integrates three core agents: the task dispatch agent, the conceptualization agent, and the modeling agent. They collaboratively achieve two objectives. First, it enhances concise initial scene descriptions, evolving them into detailed forms while dynamically adapting the text based on subsequent instructions. Second, it integrates procedural generation, extracting parameter values from enriched text to effortlessly interface with 3D software for asset creation. Our empirical investigations confirm that 3D-GPT not only interprets and executes instructions, delivering reliable results but also collaborates effectively with human designers. Furthermore, it seamlessly integrates with Blender, unlocking expanded manipulation possibilities. Our work highlights the potential of LLMs in 3D modeling, offering a basic framework for future advancements in scene generation and animation.

翻译：在追求高效自动化内容创作的过程中，程序化生成凭借可调参数与规则系统，展现出广阔前景。然而，该方法因涉及复杂的规则、算法和参数理解，对实施要求较高。为降低工作负担，我们提出3D-GPT——一个利用大型语言模型进行指令驱动三维建模的框架。该框架将大型语言模型定位为高效问题解决者，将程序化三维建模任务分解为可操作模块，并为每个任务分配最适宜的智能体。3D-GPT整合三大核心智能体：任务分配智能体、概念化智能体与建模智能体。三者协同实现两项目标：其一，优化初始场景描述的简洁性，逐步形成详尽的表述，同时根据后续指令动态调整文本；其二，集成程序化生成机制，通过从扩充文本中提取参数值，无缝对接三维软件进行资产创建。实验验证表明，3D-GPT不仅能解释并执行指令以输出可靠结果，还能与人类设计师高效协作。此外，该框架可与Blender无缝集成，拓展了三维操作的更多可能性。本工作凸显了大型语言模型在三维建模领域的潜力，为未来场景生成与动画发展提供了基础框架。