Multi-view video reconstruction plays a vital role in computer vision, enabling applications in film production, virtual reality, and motion analysis. While recent advances such as 4D Gaussian Splatting (4DGS) have demonstrated impressive capabilities in dynamic scene reconstruction, they typically rely on the assumption that input video streams are temporally synchronized. However, in real-world scenarios, this assumption often fails due to factors like camera trigger delays or independent recording setups, leading to temporal misalignment across views and reduced reconstruction quality. To address this challenge, a novel temporal alignment strategy is proposed for high-quality 4DGS reconstruction from unsynchronized multi-view videos. Our method features a coarse-to-fine alignment module that estimates and compensates for each camera's time shift. The method first determines a coarse, frame-level offset and then refines it to achieve sub-frame accuracy. This strategy can be integrated as a readily integrable module into existing 4DGS frameworks, enhancing their robustness when handling asynchronous data. Experiments show that our approach effectively processes temporally misaligned videos and significantly enhances baseline methods.


翻译:多视角视频重建在计算机视觉中扮演着至关重要的角色,广泛应用于电影制作、虚拟现实和运动分析等领域。尽管近年来如4D高斯泼溅(4DGS)等技术在动态场景重建方面展现出卓越能力,但它们通常依赖于输入视频流在时间上同步的假设。然而,在实际场景中,由于相机触发延迟或独立录制设置等因素,这一假设往往难以成立,导致视角间存在时间错位并降低重建质量。为应对这一挑战,本文提出了一种新颖的时间对齐策略,用于从非同步多视角视频中实现高质量的4DGS重建。该方法采用由粗到精的对齐模块,估计并补偿每个相机的时间偏移。首先确定粗粒度的帧级偏移,随后进行细化以达到亚帧精度。该策略可作为即插即用模块集成到现有4DGS框架中,显著提升处理异步数据时的鲁棒性。实验表明,本方法能有效处理时间错位视频,并显著提升基线方法的性能。

0
下载
关闭预览

相关内容

Integration:Integration, the VLSI Journal。 Explanation:集成,VLSI杂志。 Publisher:Elsevier。 SIT:http://dblp.uni-trier.de/db/journals/integration/
Top
微信扫码咨询专知VIP会员