In this paper, we propose an approach to address the problem of 3D reconstruction of scenes from a single image captured by a light-field camera equipped with a rolling shutter sensor. Our method leverages the 3D information cues present in the light-field and the motion information provided by the rolling shutter effect. We present a generic model for the imaging process of this sensor and a two-stage algorithm that minimizes the re-projection error while considering the position and motion of the camera in a motion-shape bundle adjustment estimation strategy. Thereby, we provide an instantaneous 3D shape-and-pose-and-velocity sensing paradigm. To the best of our knowledge, this is the first study to leverage this type of sensor for this purpose. We also present a new benchmark dataset composed of different light-fields showing rolling shutter effects, which can be used as a common base to improve the evaluation and tracking the progress in the field. We demonstrate the effectiveness and advantages of our approach through several experiments conducted for different scenes and types of motions. The source code and dataset are publicly available at: https://github.com/ICB-Vision-AI/RSLF

翻译：本文提出一种方法，以解决由配备卷帘快门传感器的光场相机所捕获的单张图像进行场景三维重建的问题。该方法利用光场中的三维信息线索以及卷帘快门效应提供的运动信息。我们为该传感器的成像过程构建了一个通用模型，并设计了一种两阶段算法，该算法在运动-形状束调整估计策略中考虑相机的位置与运动，从而最小化重投影误差。由此，我们提供了一种瞬时的三维形状-位姿-速度感知范式。据我们所知，这是首个利用此类传感器实现该目标的研究。此外，我们还提供了一个新的基准数据集，包含展现卷帘快门效应的多种光场，可作为公共基础以改进领域内的评估与进展追踪。通过针对不同场景及运动类型的多项实验，我们展示了该方法的有效性与优势。源代码与数据集公开于：https://github.com/ICB-Vision-AI/RSLF