In this position paper, we propose researching the combination of Augmented Reality (AR) and Artificial Intelligence (AI) to support conversations, inspired by the interfaces of dialogue systems commonly found in videogames. AR-capable devices are becoming more powerful and conventional in looks, as seen in head-mounted displays (HMDs) like the Snapchat Spectacles, the XREAL glasses, or the recently presented Meta Orion. This development reduces possible ergonomic, appearance, and runtime concerns, thus allowing a more straightforward integration and extended use of AR in our everyday lives, both in private and at work. At the same time, we can observe an immense surge in AI development (also at CHI). Recently notorious Large Language Models (LLMs) like OpenAI's o3-mini or DeepSeek-R1 soar over their precursors in their ability to sustain conversations, provide suggestions, and handle complex topics in (almost) real time. In combination with natural language recognition systems, which are nowadays a standard component of smartphones and similar devices (including modern AR-HMDs), it is easy to imagine a combined system that integrates into daily conversations and provides various types of assistance. Such a system would enable many opportunities for research in AR+AI, which, as stated by Hirzle et al., remains scarce. In the following, we describe how the design of a conversational AR+AI system can learn from videogame dialogue systems, and we propose use cases and research questions that can be investigated thanks to this AR+AI combination.
翻译:在本立场论文中,我们提出研究增强现实(AR)与人工智能(AI)的结合以支持对话,其灵感来源于电子游戏中常见的对话系统界面。支持AR的设备正变得功能更强大且外观更常规化,例如Snapchat Spectacles、XREAL眼镜或近期发布的Meta Orion等头戴式显示器(HMD)。这一发展减少了可能的人体工学、外观和运行时间方面的顾虑,从而使AR能更直接地融入并扩展应用于我们的日常生活和工作场景。与此同时,我们可以观察到AI领域(在CHI会议上亦是如此)正经历巨大的发展浪潮。近期备受关注的大型语言模型(LLM),如OpenAI的o3-mini或DeepSeek-R1,在维持对话、提供建议以及(近乎)实时处理复杂话题的能力上远超其前代模型。结合自然语言识别系统(现已成为智能手机及类似设备,包括现代AR-HMD的标准组件),不难设想一个能融入日常对话并提供多种形式辅助的整合系统。这样的系统将为AR+AI领域带来许多研究机遇,正如Hirzle等人所指出的,该领域的研究目前仍显不足。下文我们将阐述对话式AR+AI系统的设计如何从电子游戏对话系统中汲取经验,并提出可借助此AR+AI组合进行探索的应用场景与研究问题。