Being data-driven is one of the most iconic properties of deep learning algorithms. The birth of ImageNet drives a remarkable trend of "learning from large-scale data" in computer vision. Pretraining on ImageNet to obtain rich universal representations has been manifested to benefit various 2D visual tasks, and becomes a standard in 2D vision. However, due to the laborious collection of real-world 3D data, there is yet no generic dataset serving as a counterpart of ImageNet in 3D vision, thus how such a dataset can impact the 3D community is unraveled. To remedy this defect, we introduce MVImgNet, a large-scale dataset of multi-view images, which is highly convenient to gain by shooting videos of real-world objects in human daily life. It contains 6.5 million frames from 219,188 videos crossing objects from 238 classes, with rich annotations of object masks, camera parameters, and point clouds. The multi-view attribute endows our dataset with 3D-aware signals, making it a soft bridge between 2D and 3D vision. We conduct pilot studies for probing the potential of MVImgNet on a variety of 3D and 2D visual tasks, including radiance field reconstruction, multi-view stereo, and view-consistent image understanding, where MVImgNet demonstrates promising performance, remaining lots of possibilities for future explorations. Besides, via dense reconstruction on MVImgNet, a 3D object point cloud dataset is derived, called MVPNet, covering 87,200 samples from 150 categories, with the class label on each point cloud. Experiments show that MVPNet can benefit the real-world 3D object classification while posing new challenges to point cloud understanding. MVImgNet and MVPNet will be publicly available, hoping to inspire the broader vision community.

翻译：数据驱动是深度学习算法最显著的特性之一。ImageNet的诞生催生了计算机视觉领域“从大规模数据中学习”的显著趋势。在ImageNet上预训练以获取丰富的通用表征已被证明有利于各类2D视觉任务，并已成为2D视觉领域的标准范式。然而，由于真实世界3D数据的采集过程繁琐耗时，目前尚缺乏可作为ImageNet在3D视觉领域对应物的通用数据集，因此此类数据集对3D领域的影响尚未得到充分揭示。为弥补这一缺陷，我们提出了MVImgNet——一个大规模多视角图像数据集，该数据集通过拍摄人类日常生活中真实世界物体的视频即可便捷获取。MVImgNet包含来自219,188个视频的650万帧图像，涵盖238个物体类别，并附有物体掩码、相机参数和点云等丰富标注。多视角属性赋予该数据集3D感知信号，使其成为连接2D与3D视觉的柔性桥梁。我们通过系列探索性研究，在辐射场重建、多视角立体视觉和视角一致图像理解等多种3D与2D视觉任务上验证了MVImgNet的潜力，实验表明该数据集展现了优良性能，并为未来探索保留了广阔空间。此外，通过对MVImgNet进行密集重建，我们衍生出名为MVPNet的3D物体点云数据集，包含来自150个类别的87,200个样本，每个点云附有类别标签。实验表明，MVPNet不仅有助于真实世界3D物体分类，还向点云理解提出了新挑战。MVImgNet与MVPNet将公开发布，以期启迪更广泛的视觉研究领域。