Detecting unseen instances based on multi-view templates is a challenging problem due to its open-world nature. Traditional methodologies, which primarily rely on 2D representations and matching techniques, are often inadequate in handling pose variations and occlusions. To solve this, we introduce VoxDet, a pioneer 3D geometry-aware framework that fully utilizes the strong 3D voxel representation and reliable voxel matching mechanism. VoxDet first ingeniously proposes template voxel aggregation (TVA) module, effectively transforming multi-view 2D images into 3D voxel features. By leveraging associated camera poses, these features are aggregated into a compact 3D template voxel. In novel instance detection, this voxel representation demonstrates heightened resilience to occlusion and pose variations. We also discover that a 3D reconstruction objective helps to pre-train the 2D-3D mapping in TVA. Second, to quickly align with the template voxel, VoxDet incorporates a Query Voxel Matching (QVM) module. The 2D queries are first converted into their voxel representation with the learned 2D-3D mapping. We find that since the 3D voxel representations encode the geometry, we can first estimate the relative rotation and then compare the aligned voxels, leading to improved accuracy and efficiency. Exhaustive experiments are conducted on the demanding LineMod-Occlusion, YCB-video, and the newly built RoboTools benchmarks, where VoxDet outperforms various 2D baselines remarkably with 20% higher recall and faster speed. To the best of our knowledge, VoxDet is the first to incorporate implicit 3D knowledge for 2D tasks.

翻译：基于多视角模板检测未见实例是一项具有挑战性的问题，其根源在于开放世界特性。传统方法主要依赖二维表征与匹配技术，往往难以应对姿态变化与遮挡。为解决此问题，我们提出VoxDet——一种开创性的三维几何感知框架，该框架充分利用了强大的三维体素表征与可靠的体素匹配机制。VoxDet首先创新性地提出模板体素聚合（TVA）模块，将多视角二维图像高效转化为三维体素特征。通过关联相机位姿，这些特征被聚合为紧凑的三维模板体素。在新型实例检测中，该体素表征展现出更强的抗遮挡与姿态变化鲁棒性。我们还发现，三维重建目标有助于预训练TVA中的二维-三维映射。其次，为快速对齐模板体素，VoxDet引入查询体素匹配（QVM）模块。二维查询先通过已学习的二维-三维映射转换为体素表征。我们观察到，由于三维体素特征编码了几何信息，可先估计相对旋转再比对对齐后的体素，从而提升精度与效率。在极具挑战性的LineMod-Occlusion、YCB-video及新构建的RoboTools基准测试中，VoxDet显著优于各类二维基线模型，召回率提升20%且速度更快。据我们所知，VoxDet是首个将隐式三维知识融入二维任务的工作。