Traditional Task and Motion Planning (TAMP) systems depend on physics models for motion planning and discrete symbolic models for task planning. Although physics model are often available, symbolic models (consisting of symbolic state interpretation and action models) must be meticulously handcrafted or learned from labeled data. This process is both resource-intensive and constrains the solution to the specific domain, limiting scalability and adaptability. On the other hand, Visual Language Models (VLMs) show desirable zero-shot visual understanding (due to their extensive training on heterogeneous data), but still achieve limited planning capabilities. Therefore, integrating VLMs with classical planning for long-horizon reasoning in TAMP problems offers high potential. Recent works in this direction still lack generality and depend on handcrafted, task-specific solutions, e.g. describing all possible objects in advance, or using symbolic action models. We propose a framework that generalizes well to unseen problem instances. The method requires only lifted predicates describing relations among objects and uses VLMs to ground them from images to obtain the symbolic state. Planning is performed with domain-independent heuristic search using goal-count and width-based heuristics, without need for action models. Symbolic search over VLM-grounded state-space outperforms direct VLM-based planning and performs on par with approaches that use a VLM-derived heuristic. This shows that domain-independent search can effectively solve problems across domains with large combinatorial state spaces. We extensively evaluate on extensively evaluate our method and achieve state-of-the-art results on the ProDG and ViPlan benchmarks.

翻译：摘要：传统任务与运动规划系统依赖物理模型进行运动规划，并依赖离散符号模型进行任务规划。尽管物理模型通常可得，但符号模型（由符号状态解释和动作模型组成）必须精心手工设计或从标注数据中学习。这一过程既耗费资源，又限制了解决方案的特定领域适用性，阻碍了可扩展性和适应性。另一方面，视觉语言模型因在异构数据上的广泛训练而展现出理想的零样本视觉理解能力，但其规划能力仍有限。因此，将视觉语言模型与经典规划相结合以解决任务与运动规划问题中的长时域推理具有巨大潜力。此方向的近期工作仍缺乏通用性，依赖手工设计的任务特定方案，例如预先描述所有可能的物体或使用符号动作模型。我们提出一个能良好泛化至未见问题实例的框架。该方法仅需描述物体间关系的提升谓词，并利用视觉语言模型从图像中定位这些谓词以获取符号状态。规划通过基于目标计数和宽度启发式的领域无关启发式搜索进行，无需动作模型。基于视觉语言模型定位状态空间的符号搜索优于直接基于视觉语言模型的规划，其性能与使用视觉语言模型衍生启发式的方法相当。这表明领域无关搜索能有效解决具有大规模组合状态空间的跨领域问题。我们通过广泛评估在ProDG和ViPlan基准上取得了当前最优结果。