The current advances in generative AI for learning large neural network models with the capability to produce essays, images, music and even 3D assets from text prompts create opportunities for a manifold of disciplines. In the present paper, we study the potential of deep text-to-3D models in the engineering domain, with focus on the chances and challenges when integrating and interacting with 3D assets in computational simulation-based design optimization. In contrast to traditional design optimization of 3D geometries that often searches for the optimum designs using numerical representations, such as B-Spline surface or deformation parameters in vehicle aerodynamic optimization, natural language challenges the optimization framework by requiring a different interpretation of variation operators while at the same time may ease and motivate the human user interaction. Here, we propose and realize a fully automated evolutionary design optimization framework using Shap-E, a recently published text-to-3D asset network by OpenAI, in the context of aerodynamic vehicle optimization. For representing text prompts in the evolutionary optimization, we evaluate (a) a bag-of-words approach based on prompt templates and Wordnet samples, and (b) a tokenisation approach based on prompt templates and the byte pair encoding method from GPT4. Our main findings from the optimizations indicate that, first, it is important to ensure that the designs generated from prompts are within the object class of application, i.e. diverse and novel designs need to be realistic, and, second, that more research is required to develop methods where the strength of text prompt variations and the resulting variations of the 3D designs share causal relations to some degree to improve the optimization.

翻译：当前，生成式人工智能在学习大规模神经网络模型方面取得了进展，这类模型能够根据文本提示生成文章、图像、音乐乃至3D资产，为众多学科领域带来了机遇。本文探讨了深度文本转3D模型在工程领域的潜力，重点分析了在基于计算仿真的设计优化中集成与交互3D资产时面临的机遇与挑战。与传统的3D几何设计优化（如车辆空气动力学优化中常通过数值表示，例如B样条曲面或变形参数来搜索最优设计）不同，自然语言对优化框架提出了新的挑战，它要求对变异算子进行不同的解释，同时可能简化并激发用户交互。在此，我们提出并实现了一个全自动的进化设计优化框架，该框架采用OpenAI近期发布的文本转3D资产网络Shap-E，应用于车辆空气动力学优化。为了在进化优化中表示文本提示，我们评估了两种方法：（a）基于提示模板和Wordnet样本的词袋方法，以及（b）基于提示模板和GPT4字节对编码方法的标记化方法。优化过程中得到的主要发现表明：首先，确保由提示生成的设计属于应用对象类别至关重要，即多样且新颖的设计需具备现实性；其次，需要进一步研究开发方法，使文本提示变化的强度与3D设计产生的变化在一定程度上具有因果关系，从而改进优化效果。