Robotic branch pruning is a significantly growing research area to cope with the shortage of labor force in the context of agriculture. One fundamental requirement in robotic pruning is the perception of detailed geometry and topology of branches. However, the point clouds obtained in agricultural settings often exhibit incompleteness due to several constraints, thereby restricting the accuracy of downstream robotic pruning. In this work, we addressed the issue of point cloud quality through a simulation-based deep neural network, leveraging a Real-to-Simulation (Real2Sim) data generation pipeline that not only eliminates the need for manual parameterization but also guarantees the realism of simulated data. The simulation-based neural network was applied to jointly perform point cloud completion and skeletonization on real-world partial branches, without additional real-world training. The Sim2Real qualitative completion and skeletonization results showed the model's remarkable capability for geometry reconstruction and topology prediction. Additionally, we quantitatively evaluated the Sim2Real performance by comparing branch-level trait characterization errors using raw incomplete data and complete data. The Mean Absolute Error (MAE) reduced by 75% and 8% for branch diameter and branch angle estimation, respectively, using the best complete data, which indicates the effectiveness of the Real2Sim data in a zero-shot generalization setting. The characterization improvements contributed to the precision and efficacy of robotic branch pruning.
翻译:机器人树枝修剪是一个显著增长的研究领域,旨在应对农业劳动力短缺问题。机器人修剪的基本需求之一是感知树枝的详细几何形态与拓扑结构。然而,农业场景中获取的点云常因多种限制而存在不完整性,从而制约下游机器人修剪的精度。本研究通过基于仿真的深度神经网络解决点云质量问题,利用"真实到仿真"(Real2Sim)数据生成流程,既消除了手动参数化需求,又保证了仿真数据的真实性。该仿真神经网络无需额外真实数据训练,即可联合完成真实世界局部树枝的点云补全与骨架提取。Sim2Real定性的补全与骨架提取结果表明,该模型在几何重建与拓扑预测方面具有卓越能力。此外,我们通过对比使用原始不完整数据与完整数据时的树枝级特征表征误差,对Sim2Real性能进行了定量评估。采用最优完整数据后,枝径与枝角估计的平均绝对误差(MAE)分别降低75%和8%,这验证了Real2Sim数据在零样本泛化场景中的有效性。特征表征的改进提升了机器人树枝修剪的精度与效率。