This paper proposes "3Dify," a procedural 3D computer graphics (3D-CG) generation framework utilizing Large Language Models (LLMs). The framework enables users to generate 3D-CG content solely through natural language instructions. 3Dify is built upon Dify, an open-source platform for AI application development, and incorporates several state-of-the-art LLM-related technologies such as the Model Context Protocol (MCP) and Retrieval-Augmented Generation (RAG). For 3D-CG generation support, 3Dify automates the operation of various Digital Content Creation (DCC) tools via MCP. When DCC tools do not support MCP-based interaction, the framework employs the Computer-Using Agent (CUA) method to automate Graphical User Interface (GUI) operations. Moreover, to enhance image generation quality, 3Dify allows users to provide feedback by selecting preferred images from multiple candidates. The LLM then learns variable patterns from these selections and applies them to subsequent generations. Furthermore, 3Dify supports the integration of locally deployed LLMs, enabling users to utilize custom-developed models and to reduce both time and monetary costs associated with external API calls by leveraging their own computational resources.

翻译：本文提出“3Dify”——一个利用大型语言模型（LLMs）的程序化三维计算机图形（3D-CG）生成框架。该框架允许用户仅通过自然语言指令即可生成3D-CG内容。3Dify基于开源AI应用开发平台Dify构建，并整合了多项前沿的LLM相关技术，例如模型上下文协议（MCP）与检索增强生成（RAG）。为支持3D-CG生成，3Dify通过MCP实现了对各种数字内容创作（DCC）工具的自动化操作。当DCC工具不支持基于MCP的交互时，框架采用计算机使用代理（CUA）方法来自动化图形用户界面（GUI）操作。此外，为提升图像生成质量，3Dify允许用户通过从多个候选图像中选择偏好的图像来提供反馈。随后，LLM将从这些选择中学习变量模式，并将其应用于后续的生成过程。进一步地，3Dify支持集成本地部署的LLMs，使用户能够利用自行开发的模型，并通过调用自有计算资源来降低因使用外部API而产生的时间与经济成本。