Recent developments in 3D generative AI enable users to create bespoke 3D models from text or image prompts. However, these approaches provide limited control over spatial structure, making them ill suited for tasks requiring precise geometric composition. We present MiXR, an XR system for in-situ compositional modeling that enables users to create new 3D models by harvesting geometry from their environment. Users extract segments from captured objects and assemble new artifacts through direct 3D manipulation, while generative AI synthesizes a coherent model from the user-defined composition. This hybrid workflow allows users to define spatial structure explicitly while delegating geometric refinement to generative models, enabling them to specify spatial intent that is difficult to express through verbal prompts alone. In a controlled user study ($N=12$), participants using MiXR rated their designs as significantly closer to the target, felt more in control, and experienced lower cognitive workload compared to a generative composition baseline.
翻译:近期3D生成式AI的发展使得用户能够根据文本或图像提示创建定制化的3D模型。然而,这些方法对空间结构的控制能力有限,难以满足需要精确几何构成的任务需求。我们提出MiXR——一个面向原位组合式建模的扩展现实(XR)系统,使用户能够从环境中采集几何结构来创建新的3D模型。用户从捕获的物体中提取片段,通过直接3D操作组装新构件,同时生成式AI根据用户定义的组合合成连贯的模型。这种混合工作流程允许用户明确指定空间结构,同时将几何细化工作委托给生成模型,从而能够表达那些难以通过文字提示准确描述的空间意图。在受控用户研究($N=12$)中,与生成式组合基线方法相比,使用MiXR的参与者认为其设计更接近目标、感受到更强的控制力,且认知负荷更低。