Text-to-image AI are capable of generating novel images for inspiration, but their applications for 3D design workflows and how designers can build 3D models using AI-provided inspiration have not yet been explored. To investigate this, we integrated DALL-E, GPT-3, and CLIP within a CAD software in 3DALL-E, a plugin that generates 2D image inspiration for 3D design. 3DALL-E allows users to construct text and image prompts based on what they are modeling. In a study with 13 designers, we found that designers saw great potential in 3DALL-E within their workflows and could use text-to-image AI to produce reference images, prevent design fixation, and inspire design considerations. We elaborate on prompting patterns observed across 3D modeling tasks and provide measures of prompt complexity observed across participants. From our findings, we discuss how 3DALL-E can merge with existing generative design workflows and propose prompt bibliographies as a form of human-AI design history.

翻译：文本到图像AI能够生成新颖图像以激发灵感，但其在3D设计工作流中的应用，以及设计师如何利用AI提供的灵感构建3D模型，尚未得到探索。为研究这一问题，我们将DALL-E、GPT-3和CLIP集成到CAD软件中，开发了3DALL-E插件，该插件可为3D设计生成二维图像灵感。3DALL-E允许用户基于当前建模对象构建文本和图像提示。在针对13名设计师的研究中，我们发现设计师认为3DALL-E在其工作流中具有巨大潜力，可利用文本到图像AI生成参考图像、避免设计定势并激发设计思考。我们详细阐述了在3D建模任务中观察到的提示模式，并提供了参与者提示复杂度的度量指标。基于研究结果，我们探讨了3DALL-E如何与现有生成式设计工作流融合，并提出将提示文献库作为一种人机协同设计历史记录。