Robotic crop phenotyping has emerged as a key technology to assess crops' morphological and physiological traits at scale. These phenotypical measurements are essential for developing new crop varieties with the aim of increasing productivity and dealing with environmental challenges such as climate change. However, developing and deploying crop phenotyping robots face many challenges such as complex and variable crop shapes that complicate robotic object detection, dynamic and unstructured environments that baffle robotic control, and real-time computing and managing big data that challenge robotic hardware/software. This work specifically tackles the first challenge by proposing a novel Digital-Twin(DT)MARS-CycleGAN model for image augmentation to improve our Modular Agricultural Robotic System (MARS)'s crop object detection from complex and variable backgrounds. Our core idea is that in addition to the cycle consistency losses in the CycleGAN model, we designed and enforced a new DT-MARS loss in the deep learning model to penalize the inconsistency between real crop images captured by MARS and synthesized images sensed by DT MARS. Therefore, the generated synthesized crop images closely mimic real images in terms of realism, and they are employed to fine-tune object detectors such as YOLOv8. Extensive experiments demonstrated that our new DT/MARS-CycleGAN framework significantly boosts our MARS' crop object/row detector's performance, contributing to the field of robotic crop phenotyping.

翻译：机器人作物表型分析已成为规模化评估作物形态与生理性状的关键技术。这些表型测量对于开发旨在提升产量并应对气候变化等环境挑战的新品种至关重要。然而，作物表型机器人的开发与部署面临诸多挑战，例如复杂多变的作物形态增加了机器人目标检测的难度，动态非结构化环境干扰了机器人控制，实时计算与大数据管理对机器人软硬件提出了更高要求。本研究聚焦于第一个挑战，提出了一种新型数字孪生（DT）MARS-CycleGAN模型用于图像增强，以改进我们的模块化农业机器人系统（MARS）在复杂多变背景下的作物目标检测能力。核心思想是：在CycleGAN模型的循环一致性损失基础上，我们设计并引入了一个新的DT-MARS损失函数，用于惩罚MARS采集的真实作物图像与DT MARS感知的合成图像之间的不一致性。因此，生成的合成作物图像在逼真度上高度模拟真实图像，并用于微调YOLOv8等目标检测器。大量实验表明，我们提出的DT/MARS-CycleGAN框架显著提升了MARS作物/行检测器的性能，为机器人作物表型分析领域做出了贡献。