While Cave Automatic Virtual Environment (CAVE) systems have long enabled room-scale virtual reality and various kinds of interactivity, their content has largely remained predetermined. We present \textit{Storycaster}, a generative AI CAVE system that transforms physical rooms into responsive storytelling environments. Unlike headset-based VR, \textit{Storycaster} preserves spatial awareness, using live camera feeds to augment the walls with cylindrical projections, allowing users to create worlds that blend with their physical surroundings. Additionally, our system enables object-level editing, where physical items in the room can be transformed to their virtual counterparts in a story. A narrator agent guides participants, enabling them to co-create stories that evolve in response to voice commands, with each scene enhanced by generated ambient audio, dialogue, and imagery. Participants in our study ($n=13$) found the system highly immersive and engaging, with narrator and audio most impactful, while also highlighting areas for improvement in latency and image resolution.
翻译:尽管洞穴自动虚拟环境(CAVE)系统长期以来能够实现房间规模的虚拟现实和各种交互形式,但其内容在很大程度上仍是预先确定的。我们提出了 \textit{Storycaster},一种生成式人工智能 CAVE 系统,可将物理房间转变为响应式的叙事环境。与基于头戴式设备的虚拟现实不同,\textit{Storycaster} 保持了空间感知能力,利用实时摄像头画面通过柱面投影增强墙壁,使用户能够创建与物理环境融合的世界。此外,我们的系统支持物体级编辑,房间内的物理物品可以在故事中被转化为其虚拟对应物。一个旁白代理引导参与者,使他们能够通过语音指令共同创作动态发展的故事,每个场景都通过生成的氛围音频、对话和图像得到增强。我们研究中的参与者($n=13$)认为该系统具有高度的沉浸感和吸引力,其中旁白和音频效果最为突出,同时也指出了在延迟和图像分辨率方面有待改进之处。