Mixed Reality (MR) aims to blend digital and physical worlds, but the absence of haptic feedback often breaks visual-tactile consistency. We introduce Prop-Chromeleon, a MR system based on generative artificial intelligence (AI) that dynamically transforms everyday objects into adaptive passive haptic props through user-provided text prompts. Our AI pipeline performs generation and anchoring of virtual assets that align with the shape of physical props, allowing us to study how virtual content generation behaves under geometric and prompt-based constraints. We evaluate Prop-Chromeleon's effectiveness through a generation study using varied object shapes and user prompts, combining quantitative shape similarity metrics with qualitative prompt fidelity analysis. Our user study further showcases Prop-Chromeleon's improvements in perceived realism, immersion, and enjoyment compared to static baselines. These results show that shape-aware generation can support both believable haptic interaction and creative engagement in MR.
翻译:混合现实(MR)旨在融合数字与物理世界,但触觉反馈的缺失常破坏视觉-触觉一致性。我们提出Prop-Chromeleon——一种基于生成式人工智能(AI)的MR系统,可通过用户文本提示将日常物品动态转换为适应性被动触觉道具。我们的AI流水线实现了与物理道具形状匹配的虚拟资产生成与锚定,从而研究虚拟内容生成在几何形状与提示约束下的行为表现。通过采用不同物体形状与用户提示的生成研究,结合定量形状相似性指标与定性提示保真度分析,我们评估了Prop-Chromeleon的有效性。用户研究进一步表明,相比静态基线方法,Prop-Chromeleon在感知真实感、沉浸感与愉悦度方面均有提升。这些结果表明,形状感知生成能够支撑MR中可信的触觉交互与创造性参与。