Reconstructing fast-dynamic scenes from multi-view videos is crucial for high-speed motion analysis and realistic 4D reconstruction. However, the majority of 4D capture systems are limited to frame rates below 30 FPS (frames per second), and a direct 4D reconstruction of high-speed motion from low FPS input may lead to undesirable results. In this work, we propose a high-speed 4D capturing system only using low FPS cameras, through novel capturing and processing modules. On the capturing side, we propose an asynchronous capture scheme that increases the effective frame rate by staggering the start times of cameras. By grouping cameras and leveraging a base frame rate of 25 FPS, our method achieves an equivalent frame rate of 100-200 FPS without requiring specialized high-speed cameras. On processing side, we also propose a novel generative model to fix artifacts caused by 4D sparse-view reconstruction, as asynchrony reduces the number of viewpoints at each timestamp. Specifically, we propose to train a video-diffusion-based artifact-fix model for sparse 4D reconstruction, which refines missing details, maintains temporal consistency, and improves overall reconstruction quality. Experimental results demonstrate that our method significantly enhances high-speed 4D reconstruction compared to synchronous capture.

翻译：从多视角视频中重建快速动态场景对于高速运动分析和逼真的四维重建至关重要。然而，大多数四维采集系统的帧率限制在30 FPS（帧/秒）以下，直接从低帧率输入进行高速运动的四维重建可能导致不理想的结果。本研究提出一种仅使用低帧率相机的高速四维采集系统，通过新颖的采集与处理模块实现。在采集端，我们提出异步采集方案，通过错开各相机的起始时间来提高有效帧率。通过分组相机并利用25 FPS的基础帧率，我们的方法无需专用高速相机即可实现等效100-200 FPS的帧率。在处理端，针对因异步性导致每个时间戳视角数量减少而产生的四维稀疏视角重建伪影，我们提出一种新型生成模型进行修复。具体而言，我们训练基于视频扩散的伪影修复模型用于稀疏四维重建，该模型能精化缺失细节、保持时间一致性并提升整体重建质量。实验结果表明，与同步采集相比，我们的方法显著增强了高速四维重建效果。