High-quality scene reconstruction and novel view synthesis based on Gaussian Splatting (3DGS) typically require steady, high-quality photographs, often impractical to capture with handheld cameras. We present a method that adapts to camera motion and allows high-quality scene reconstruction with handheld video data suffering from motion blur and rolling shutter distortion. Our approach is based on detailed modelling of the physical image formation process and utilizes velocities estimated using visual-inertial odometry (VIO). Camera poses are considered non-static during the exposure time of a single image frame and camera poses are further optimized in the reconstruction process. We formulate a differentiable rendering pipeline that leverages screen space approximation to efficiently incorporate rolling-shutter and motion blur effects into the 3DGS framework. Our results with both synthetic and real data demonstrate superior performance in mitigating camera motion over existing methods, thereby advancing 3DGS in naturalistic settings.

翻译：基于高斯溅射（3DGS）的高质量场景重建与新视角合成通常需要稳定、高质量的照片，而这在使用手持相机拍摄时往往难以实现。本文提出一种适应相机运动的方法，能够利用存在运动模糊与卷帘快门失真的手持视频数据实现高质量场景重建。该方法基于对物理成像过程的精细建模，并利用视觉惯性里程计（VIO）估计的速度信息。在单帧图像的曝光时间内，相机位姿被视为动态变化，并在重建过程中进一步优化。我们构建了一个可微分渲染管线，通过屏幕空间近似将卷帘快门与运动模糊效应高效整合到3DGS框架中。合成数据与真实数据的实验结果表明，本方法在消除相机运动影响方面优于现有技术，从而推动了3DGS在自然拍摄场景中的应用。