In mixed reality (MR) environments, understanding space and creating virtual objects is crucial to providing an intuitive and rich user experience. This paper introduces sMoRe (Spatial Mapping and Object Rendering Environment), an MR application that combines Generative AI (GenAI) with large language models (LLMs) to assist users in creating, placing, and managing virtual objects within physical spaces. sMoRe allows users to use voice or typed text commands to create and place virtual objects using GenAI while specifying spatial constraints. The system leverages LLMs to interpret users' commands, analyze the current scene, and identify optimal locations. Additionally, sMoRe integrates text-to-3D generative AI to dynamically create 3D objects based on users' descriptions. Our user study demonstrates the effectiveness of sMoRe in enhancing user comprehension, interaction, and organization of the MR environment.

翻译：在混合现实（MR）环境中，理解空间并创建虚拟物体对于提供直观且丰富的用户体验至关重要。本文介绍了sMoRe（空间映射与物体渲染环境），这是一款结合生成式人工智能（GenAI）与大语言模型（LLMs）的MR应用，旨在协助用户在物理空间内创建、放置和管理虚拟物体。sMoRe允许用户通过语音或文本输入指令，利用GenAI创建并放置虚拟物体，同时指定空间约束条件。该系统借助LLMs解析用户指令、分析当前场景并确定最优位置。此外，sMoRe集成了文本到3D生成式人工智能技术，可根据用户描述动态生成3D物体。我们的用户研究证明了sMoRe在提升用户对MR环境的理解、交互与组织方面的有效性。