We introduce Memento, a conversational AR assistant that permanently captures and memorizes user's verbal queries alongside their spatiotemporal and activity contexts. By storing these "memories," Memento discovers connections between users' recurring interests and the contexts that trigger them. Upon detection of similar or identical spatiotemporal activity, Memento proactively recalls user interests and delivers up-to-date responses through AR, seamlessly integrating AR experience into their daily routine. Unlike prior work, each interaction in Memento is not a transient event, but a connected series of interactions with coherent long--term perspective, tailored to the user's broader multimodal (visual, spatial, temporal, and embodied) context. We conduct preliminary evaluation through user feedbacks with participants of diverse expertise in immersive apps, and explore the value of proactive context-aware AR assistant in everyday settings. We share our findings and challenges in designing a proactive, context-aware AR system.
翻译:本文介绍Memento——一种对话式增强现实助手,能够永久捕获并记忆用户的口头查询及其时空与活动情境。通过存储这些“记忆”,Memento能够发现用户重复出现的兴趣点与触发这些兴趣的情境之间的关联。当检测到相似或完全相同的时空活动时,Memento会主动唤起用户兴趣,并通过增强现实提供最新响应,将增强现实体验无缝融入用户的日常生活。与先前研究不同,Memento中的每次交互并非孤立事件,而是具有连贯长期视角的关联交互序列,且适配用户更广泛的多模态(视觉、空间、时间与具身)情境。我们通过对沉浸式应用具有不同专业背景的参与者收集用户反馈进行初步评估,并探索主动式情境感知增强现实助手在日常环境中的价值。同时分享了我们在设计主动式情境感知增强现实系统过程中的发现与挑战。