Sound plays a crucial role in enhancing user experience and immersiveness in Augmented Reality (AR). However, current platforms lack support for AR sound authoring due to limited interaction types, challenges in collecting and specifying context information, and difficulty in acquiring matching sound assets. We present SonifyAR, an LLM-based AR sound authoring system that generates context-aware sound effects for AR experiences. SonifyAR expands the current design space of AR sound and implements a Programming by Demonstration (PbD) pipeline to automatically collect contextual information of AR events, including virtual content semantics and real world context. This context information is then processed by a large language model to acquire sound effects with Recommendation, Retrieval, Generation, and Transfer methods. To evaluate the usability and performance of our system, we conducted a user study with eight participants and created five example applications, including an AR-based science experiment, an improving case for AR headset safety, and an assisting example for low vision AR users.

翻译：声音在增强现实（AR）中对于提升用户体验和沉浸感起着至关重要的作用。然而，当前平台由于交互类型受限、情境信息收集与指定困难以及匹配声音素材获取难度大，缺乏对AR声音创作的支持。我们提出SonifyAR，一种基于大语言模型的AR声音创作系统，可为AR体验生成情境感知音效。SonifyAR拓展了现有AR声音的设计空间，并实现了一条“示范编程”（PbD）流程，用于自动收集AR事件的上下文信息，包括虚拟内容语义和现实世界情境。随后，这些上下文信息由大语言模型处理，通过推荐、检索、生成和迁移方法获取音效。为评估系统的可用性和性能，我们开展了包含八名参与者的用户研究，并创建了五个示例应用，涵盖基于AR的科学实验、提升AR头戴设备安全性的改进案例，以及为低视力AR用户提供的辅助示例。