3D reconstruction from multiple views is a successful computer vision field with multiple deployments in applications. State of the art is based on traditional RGB frames that enable optimization of photo-consistency cross views. In this paper, we study the problem of 3D reconstruction from event-cameras, motivated by the advantages of event-based cameras in terms of low power and latency as well as by the biological evidence that eyes in nature capture the same data and still perceive well 3D shape. The foundation of our hypothesis that 3D reconstruction is feasible using events lies in the information contained in the occluding contours and in the continuous scene acquisition with events. We propose Apparent Contour Events (ACE), a novel event-based representation that defines the geometry of the apparent contour of an object. We represent ACE by a spatially and temporally continuous implicit function defined in the event x-y-t space. Furthermore, we design a novel continuous Voxel Carving algorithm enabled by the high temporal resolution of the Apparent Contour Events. To evaluate the performance of the method, we collect MOEC-3D, a 3D event dataset of a set of common real-world objects. We demonstrate the ability of EvAC3D to reconstruct high-fidelity mesh surfaces from real event sequences while allowing the refinement of the 3D reconstruction for each individual event.
翻译:[从多视图进行三维重建是一个成功的计算机视觉领域,已有多项实际应用。现有技术基于传统RGB帧,能够优化跨视图的光度一致性。本文研究了从事件相机进行三维重建的问题,其动机源于事件相机在低功耗和低延迟方面的优势,以及自然界眼睛捕捉相同数据却能良好感知三维形状的生物学证据。我们认为利用事件进行三维重建可行的基础在于遮挡轮廓所含的信息以及事件的连续场景采集特性。我们提出事件表观轮廓(Apparent Contour Events, ACE),这是一种新颖的基于事件的表示方法,定义了物体表观轮廓的几何结构。我们通过在事件x-y-t空间中定义时空连续的隐函数来表示ACE。此外,我们设计了一种新颖的连续体素雕刻算法,该算法借助事件表观轮廓的高时间分辨率得以实现。为评估该方法性能,我们收集了MOEC-3D数据集,这是一个包含多种常见真实物体的三维事件数据集。我们展示了EvAC3D能够从真实事件序列重建高保真网格表面,同时允许对每个单独事件进行三维重建的细化。]