Autonomous harvesting in the open presents a complex manipulation problem. In most scenarios, an autonomous system has to deal with significant occlusion and require interaction in the presence of large structural uncertainties (every plant is different). Perceptual and modeling uncertainty make design of reliable manipulation controllers for harvesting challenging, resulting in poor performance during deployment. We present a sim2real reinforcement learning (RL) framework for occlusion-aware plant manipulation, where a policy is learned entirely in simulation to reposition stems and leaves to reveal target fruit(s). In our proposed approach, we decouple high-level kinematic planning from low-level compliant control which simplifies the sim2real transfer. This decomposition allows the learned policy to generalize across multiple plants with different stiffness and morphology. In experiments with multiple real-world plant setups, our system achieves up to 86.7% success in exposing target fruits, demonstrating robustness to occlusion variation and structural uncertainty.

翻译：开放环境下的自主采摘是一个复杂的操作问题。在大多数场景中，自主系统必须处理严重的遮挡，并且需要在存在巨大结构不确定性（每株植物都不同）的情况下进行交互。感知和建模的不确定性使得为采摘任务设计可靠的操作控制器具有挑战性，导致部署期间性能不佳。我们提出了一种用于遮挡感知植物操作的仿真到现实强化学习框架，其中策略完全在仿真环境中学习，通过重新定位茎叶以显露目标果实。在我们提出的方法中，我们将高层运动规划与低层顺应性控制解耦，从而简化了仿真到现实的迁移。这种分解使得学习到的策略能够泛化到具有不同刚度和形态的多种植物。在多个真实世界植物设置下的实验中，我们的系统在显露目标果实方面取得了高达86.7%的成功率，展示了对遮挡变化和结构不确定性的鲁棒性。