Addressing the intricate challenge of modeling and re-rendering dynamic scenes, most recent approaches have sought to simplify these complexities using plane-based explicit representations, overcoming the slow training time issues associated with methods like Neural Radiance Fields (NeRF) and implicit representations. However, the straightforward decomposition of 4D dynamic scenes into multiple 2D plane-based representations proves insufficient for re-rendering high-fidelity scenes with complex motions. In response, we present a novel direction-aware representation (DaRe) approach that captures scene dynamics from six different directions. This learned representation undergoes an inverse dual-tree complex wavelet transformation (DTCWT) to recover plane-based information. DaReNeRF computes features for each space-time point by fusing vectors from these recovered planes. Combining DaReNeRF with a tiny MLP for color regression and leveraging volume rendering in training yield state-of-the-art performance in novel view synthesis for complex dynamic scenes. Notably, to address redundancy introduced by the six real and six imaginary direction-aware wavelet coefficients, we introduce a trainable masking approach, mitigating storage issues without significant performance decline. Moreover, DaReNeRF maintains a 2x reduction in training time compared to prior art while delivering superior performance.

翻译：针对动态场景建模与重渲染中的复杂挑战，近期多数方法采用基于平面的显式表示来简化这些复杂性，从而克服了神经辐射场（NeRF）等隐式表示方法训练耗时长的问题。然而，将4D动态场景直接分解为多个2D平面表示，难以充分重渲染具有复杂运动的高保真场景。为此，我们提出一种新颖的方向感知表示（DaRe）方法，从六个不同方向捕获场景动态。该学习到的表示通过逆双树复小波变换（DTCWT）恢复平面信息。DaReNeRF通过融合这些恢复平面的向量，计算每个时空点的特征。将DaReNeRF与用于颜色回归的轻量级MLP相结合，并在训练中利用体渲染技术，可在复杂动态场景的新视角合成中达到最先进的性能。值得注意的是，为消除六个实部和六个虚部方向感知小波系数带来的冗余，我们引入了一种可训练的掩码方法，在无显著性能下降的前提下缓解存储问题。此外，与现有技术相比，DaReNeRF在保持训练时间降低2倍的同时实现了更优性能。