While recent methods for motion and stereo estimation recover an unprecedented amount of details, such highly detailed structures are neither adequately reflected in the data of existing benchmarks nor their evaluation methodology. Hence, we introduce Spring $-$ a large, high-resolution, high-detail, computer-generated benchmark for scene flow, optical flow, and stereo. Based on rendered scenes from the open-source Blender movie "Spring", it provides photo-realistic HD datasets with state-of-the-art visual effects and ground truth training data. Furthermore, we provide a website to upload, analyze and compare results. Using a novel evaluation methodology based on a super-resolved UHD ground truth, our Spring benchmark can assess the quality of fine structures and provides further detailed performance statistics on different image regions. Regarding the number of ground truth frames, Spring is 60$\times$ larger than the only scene flow benchmark, KITTI 2015, and 15$\times$ larger than the well-established MPI Sintel optical flow benchmark. Initial results for recent methods on our benchmark show that estimating fine details is indeed challenging, as their accuracy leaves significant room for improvement. The Spring benchmark and the corresponding datasets are available at http://spring-benchmark.org.
翻译:尽管近年来运动估计与立体匹配方法已能够恢复前所未有的细节信息,但现有基准测试的数据及其评估方法均未能充分反映此类高精细结构。为此,我们提出Spring——一个基于开源Blender电影《Spring》渲染场景的大规模、高分辨率、高细节计算机生成基准,涵盖场景流、光流与立体匹配。该基准提供具有最先进视觉效果与真实标注训练数据的光照逼真高清数据集,同时建立用于上传、分析及对比结果的专用网站。通过基于超分辨率UHD真实标注的新型评估方法,Spring基准能够评估精细结构的质量,并针对不同图像区域提供详细的性能统计。在真实标注帧数量上,Spring是唯一场景流基准KITTI 2015的60倍,是公认MPI Sintel光流基准的15倍。最新方法在Spring基准上的初步结果表明,精细细节的估计确实具有挑战性,其精度仍有显著提升空间。Spring基准及对应数据集可通过http://spring-benchmark.org获取。