Immersion plays a vital role when designing cinematic creations, yet the difficulty in immersive shooting prevents designers to create satisfactory outputs. In this work, we analyze the specific components that contribute to cinematographic immersion considering spatial, emotional, and aesthetic level, while these components are then combined into a high-level evaluation mechanism. Guided by such a immersion mechanism, we propose a GAN-based camera control system that is able to generate actor-driven camera movements in the 3D virtual environment to obtain immersive film sequences. The proposed encoder-decoder architecture in the generation flow transfers character motion into camera trajectory conditioned on an emotion factor. This ensures spatial and emotional immersion by performing actor-camera synchronization physically and psychologically. The emotional immersion is further strengthened by incorporating regularization that controls camera shakiness for expressing different mental statuses. To achieve aesthetic immersion, we make effort to improve aesthetic frame compositions by modifying the synthesized camera trajectory. Based on a self-supervised adjustor, the adjusted camera placements can project the character to the appropriate on-frame locations following aesthetic rules. The experimental results indicate that our proposed camera control system can efficiently offer immersive cinematic videos, both quantitatively and qualitatively, based on a fine-grained immersive shooting. Live examples are shown in the supplementary video.

翻译：沉浸感在电影创作设计中扮演着至关重要的角色，然而沉浸式拍摄的难度阻碍了设计师产出令人满意的成果。本研究从空间、情感与审美三个层面，分析了构成电影沉浸感的具体要素，并将这些要素整合为高层次评估机制。在此沉浸机制引导下，我们提出基于GAN的摄像机控制系统，能够在三维虚拟环境中生成由演员驱动的摄像机运动，从而获得具有沉浸感的电影序列。生成流程中采用的编码器-解码器架构以情感因子为条件，将角色动作转化为摄像机轨迹，通过物理与心理层面的演员-摄像机同步确保空间与情感沉浸。通过引入控制摄像机抖动程度的正则化项以表达不同心理状态，进一步强化情感沉浸。为实现审美沉浸，我们通过修正合成摄像机轨迹来提升画面构图美感。基于自监督调节器，调整后的摄像机位可将角色投射至符合美学规则的合适画面位置。实验结果表明，基于细粒度沉浸式拍摄，本研究所提摄像机控制系统无论从定量还是定性角度，均能高效生成具有沉浸感的电影视频。实际案例见补充视频。