We introduce CinemaWorld, a generative augmented reality system that augments the viewer's physical surroundings with automatically generated mixed reality 3D content extracted from and synchronized with 2D movie scenes. Our system preprocesses films to extract key features using multimodal large language models (LLMs), generates dynamic 3D augmentations with generative AI, and embeds them spatially into the viewer's physical environment on the Meta Quest 3. To explore the design space of CinemaWorld, we conducted an elicitation study with eight film students, which led us to identify several key augmentation types, including particle effects, surrounding objects, textural overlays, character-driven augmentation, and lighting effects. We evaluated our system through a technical evaluation (N=100 video clips), a user study (N=12), and expert interviews with film creators (N=8). Results indicate that CinemaWorld enhances immersion and enjoyment, suggesting its potential to enrich the film-viewing experience.

翻译：本文介绍CinemaWorld，一种生成式增强现实系统，该系统通过从二维电影场景中提取并同步的自动生成混合现实三维内容，增强观众物理环境的观影体验。本系统通过多模态大语言模型预处理影片以提取关键特征，利用生成式人工智能生成动态三维增强内容，并将其空间嵌入至Meta Quest 3设备所呈现的观众物理环境中。为探索CinemaWorld的设计空间，我们与八名电影专业学生开展了设计启发研究，从而确定了若干关键增强类型，包括粒子特效、环境物体、纹理叠加、角色驱动增强及光照效果。我们通过技术评估（N=100段视频片段）、用户研究（N=12）以及与电影创作者的专业访谈（N=8）对系统进行了评估。结果表明，CinemaWorld显著提升了沉浸感与观影愉悦度，展现了其在丰富电影观看体验方面的潜力。