The reconstruction and novel view synthesis of dynamic scenes recently gained increased attention. As reconstruction from large-scale multi-view data involves immense memory and computational requirements, recent benchmark datasets provide collections of single monocular views per timestamp sampled from multiple (virtual) cameras. We refer to this form of inputs as "monocularized" data. Existing work shows impressive results for synthetic setups and forward-facing real-world data, but is often limited in the training speed and angular range for generating novel views. This paper addresses these limitations and proposes a new method for full 360{\deg} inward-facing novel view synthesis of non-rigidly deforming scenes. At the core of our method are: 1) An efficient deformation module that decouples the processing of spatial and temporal information for accelerated training and inference; and 2) A static module representing the canonical scene as a fast hash-encoded neural radiance field. In addition to existing synthetic monocularized data, we systematically analyze the performance on real-world inward-facing scenes using a newly recorded challenging dataset sampled from a synchronized large-scale multi-view rig. In both cases, our method is significantly faster than previous methods, converging in less than 7 minutes and achieving real-time framerates at 1K resolution, while obtaining a higher visual accuracy for generated novel views. Our source code and data is available at our project page https://graphics.tu-bs.de/publications/kappel2022fast.

翻译：动态场景的重建与新视角合成近期受到越来越多的关注。由于大规模多视角数据重建涉及巨大的内存和计算需求，近年基准数据集通常提供从多个（虚拟）相机采样的每个时间戳的单张单目视图。我们将此类输入形式称为"单目化"数据。现有工作在合成场景和前向真实数据上展示了令人印象深刻的结果，但在训练速度和生成新视角的角范围上存在局限。本文针对这些局限性，提出了一种用于非刚性变形场景的全360°内视新视角合成新方法。该方法的核心包括：1）高效变形模块，通过解耦时空信息处理加速训练与推理；2）静态模块，将规范场景表示为快速哈希编码神经辐射场。除现有合成单目化数据外，我们利用从同步大规模多视角采集装置采样得到的新挑战性数据集，系统分析了在真实内视场景上的性能。在两种场景中，我们的方法均显著快于先前方法，在7分钟内收敛，并以1K分辨率实现实时帧率，同时生成的新视角具有更高的视觉精度。源代码与数据详见项目主页 https://graphics.tu-bs.de/publications/kappel2022fast。