Monitoring plants and fruits at high resolution play a key role in the future of agriculture. Accurate 3D information can pave the way to a diverse number of robotic applications in agriculture ranging from autonomous harvesting to precise yield estimation. Obtaining such 3D information is non-trivial as agricultural environments are often repetitive and cluttered, and one has to account for the partial observability of fruit and plants. In this paper, we address the problem of jointly estimating complete 3D shapes of fruit and their pose in a 3D multi-resolution map built by a mobile robot. To this end, we propose an online multi-resolution panoptic mapping system where regions of interest are represented with a higher resolution. We exploit data to learn a general fruit shape representation that we use at inference time together with an occlusion-aware differentiable rendering pipeline to complete partial fruit observations and estimate the 7 DoF pose of each fruit in the map. The experiments presented in this paper, evaluated both in the controlled environment and in a commercial greenhouse, show that our novel algorithm yields higher completion and pose estimation accuracy than existing methods, with an improvement of 41% in completion accuracy and 52% in pose estimation accuracy while keeping a low inference time of 0.6s in average.

翻译：监测植物和果实的高分辨率信息在农业的未来发展中扮演着关键角色。精确的三维数据可为农业中从自主采摘到精准产量估算等一系列机器人应用奠定基础。然而，由于农业环境通常具有重复性和杂乱性，且需考虑果实与植物的部分可观测性，获取此类三维信息并非易事。本文研究了在移动机器人构建的三维多分辨率地图中联合估计果实的完整三维形状及其位姿的问题。为此，我们提出了一种在线多分辨率全景建图系统，其中感兴趣区域以更高分辨率表示。我们利用数据学习通用的果实形状表征，在推理阶段结合遮挡感知的可微渲染管线，以补全部分果实观测信息并估计地图中每个果实的七自由度位姿。本文在受控环境与商业温室中开展的实验表明，与现有方法相比，所提出的算法在补全精度和位姿估计精度上分别提升41%和52%，同时平均推理时间保持在0.6秒的低水平。