We focus on enabling damage and tampering detection in logistics and tackle the problem of 3D shape reconstruction of potentially damaged parcels. As input we utilize single RGB images, which corresponds to use-cases where only simple handheld devices are available, e.g. for postmen during delivery or clients on delivery. We present a novel synthetic dataset, named Parcel3D, that is based on the Google Scanned Objects (GSO) dataset and consists of more than 13,000 images of parcels with full 3D annotations. The dataset contains intact, i.e. cuboid-shaped, parcels and damaged parcels, which were generated in simulations. We work towards detecting mishandling of parcels by presenting a novel architecture called CubeRefine R-CNN, which combines estimating a 3D bounding box with an iterative mesh refinement. We benchmark our approach on Parcel3D and an existing dataset of cuboid-shaped parcels in real-world scenarios. Our results show, that while training on Parcel3D enables transfer to the real world, enabling reliable deployment in real-world scenarios is still challenging. CubeRefine R-CNN yields competitive performance in terms of Mesh AP and is the only model that directly enables deformation assessment by 3D mesh comparison and tampering detection by comparing viewpoint invariant parcel side surface representations. Dataset and code are available at https://a-nau.github.io/parcel3d.

翻译：我们聚焦于物流中的损坏与篡改检测问题，并针对潜在受损包裹的三维形状重建难题展开研究。输入采用单张RGB图像，这对应于仅配备简单手持设备（如投递员在配送期间或客户在收货时）的应用场景。我们提出名为Parcel3D的新型合成数据集，该数据集基于Google扫描物体（GSO）构建，包含逾13,000张带有完整三维标注的包裹图像。数据集中包含完好（即立方体形）包裹与通过仿真生成的受损包裹。为检测包裹处理不当行为，我们提出名为CubeRefine R-CNN的新型架构，该架构将三维边界框估计与迭代网格细化相结合。我们在Parcel3D及现有真实场景立方体包裹数据集上对方法进行基准测试。结果表明，尽管基于Parcel3D的训练可实现向真实场景的迁移，但在实际环境中实现可靠部署仍具挑战。CubeRefine R-CNN在网格平均精度（Mesh AP）指标上表现优异，且是唯一能够通过三维网格对比直接实现变形评估，并通过视角不变包裹侧面表示对比实现篡改检测的模型。数据集与代码已开源：https://a-nau.github.io/parcel3d。