Pick-and-place is one of the fundamental tasks in robotics research. However, the attention has been mostly focused on the ``pick'' task, leaving the ``place'' task relatively unexplored. In this paper, we address the problem of placing objects in the context of a teleoperation framework. Particularly, we focus on two aspects of the place task: stability robustness and contextual reasonableness of object placements. Our proposed method combines simulation-driven physical stability verification via real-to-sim and the semantic reasoning capability of large language models. In other words, given place context information (e.g., user preferences, object to place, and current scene information), our proposed method outputs a probability distribution over the possible placement candidates, considering the robustness and reasonableness of the place task. Our proposed method is extensively evaluated in two simulation and one real world environments and we show that our method can greatly increase the physical plausibility of the placement as well as contextual soundness while considering user preferences.
翻译:论文摘要:抓取-放置是机器人学中的基础任务之一。然而,研究重点多集中于"抓取"任务,而"放置"任务相对较少被探索。本文针对遥操作框架中的物体放置问题展开研究,特别关注放置任务的两个核心方面:放置稳定性鲁棒性与放置情境合理性。我们提出的方法将基于仿真驱动物理稳定性验证(通过真实-仿真映射)与大语言模型的语义推理能力相结合。具体而言,给定放置情境信息(如用户偏好、待放置物体及当前场景信息),本方法可针对可能的放置候选位置输出概率分布,同时兼顾放置任务的鲁棒性与合理性。我们在两个仿真环境和一个真实世界环境中对所提方法进行了全面评估,实验结果表明,本方法在考虑用户偏好的同时,能显著提升放置的物理合理性与情境协调性。