Selective robotic harvesting is a promising technological solution to address labour shortages which are affecting modern agriculture in many parts of the world. For an accurate and efficient picking process, a robotic harvester requires the precise location and orientation of the fruit to effectively plan the trajectory of the end effector. The current methods for estimating fruit orientation employ either complete 3D information which typically requires registration from multiple views or rely on fully-supervised learning techniques, which require difficult-to-obtain manual annotation of the reference orientation. In this paper, we introduce a novel key-point-based fruit orientation estimation method allowing for the prediction of 3D orientation from 2D images directly. The proposed technique can work without full 3D orientation annotations but can also exploit such information for improved accuracy. We evaluate our work on two separate datasets of strawberry images obtained from real-world data collection scenarios. Our proposed method achieves state-of-the-art performance with an average error as low as $8^{\circ}$, improving predictions by $\sim30\%$ compared to previous work presented in~\cite{wagner2021efficient}. Furthermore, our method is suited for real-time robotic applications with fast inference times of $\sim30$ms.

翻译：选择性机器人采摘是解决影响全球许多地区现代农业的劳动力短缺问题的一种有前景的技术方案。为了实现准确且高效的采摘过程，机器人采摘器需要精确获取水果的位置和方向，以便有效规划末端执行器的轨迹。当前水果方向估计方法要么使用完整的3D信息（通常需要从多个视角进行配准），要么依赖全监督学习技术，这需要难以获取的参考方向人工标注。本文提出了一种新颖的基于关键点的水果方向估计方法，可直接从2D图像预测3D方向。该技术无需完整的3D方向标注即可工作，但也能够利用此类信息来提高精度。我们在两个来自真实数据采集场景的草莓图像数据集上评估了我们的方法。所提出的方法实现了最优性能，平均误差低至 $8^{\circ}$，与~\cite{wagner2021efficient}中提出的先前工作相比，预测精度提高了约30%。此外，我们的方法适用于实时机器人应用，推理时间仅为约30毫秒。