In this paper, we propose using deep neural architectures (i.e., vision transformers and ResNet) as heuristics for sequential decision-making in robotic manipulation problems. This formulation enables predicting the subset of objects that are relevant for completing a task. Such problems are often addressed by task and motion planning (TAMP) formulations combining symbolic reasoning and continuous motion planning. In essence, the action-object relationships are resolved for discrete, symbolic decisions that are used to solve manipulation motions (e.g., via nonlinear trajectory optimization). However, solving long-horizon tasks requires consideration of all possible action-object combinations which limits the scalability of TAMP approaches. To overcome this combinatorial complexity, we introduce a visual perception module integrated with a TAMP-solver. Given a task and an initial image of the scene, the learned model outputs the relevancy of objects to accomplish the task. By incorporating the predictions of the model into a TAMP formulation as a heuristic, the size of the search space is significantly reduced. Results show that our framework finds feasible solutions more efficiently when compared to a state-of-the-art TAMP solver.

翻译：本文提出利用深度神经架构（即视觉Transformer和ResNet）作为机器人操作问题中序列决策的启发式方法。该公式能够预测与完成任务相关的物体子集。此类问题通常通过任务与运动规划（TAMP）框架解决，该框架结合了符号推理与连续运动规划。本质上，动作-物体关系被解析为离散的符号决策，进而用于求解操作运动（例如通过非线性轨迹优化）。然而，解决长时域任务需考虑所有可能的动作-物体组合，这限制了TAMP方法的可扩展性。为克服这种组合复杂性，我们引入了一个与TAMP求解器集成的视觉感知模块。给定任务和场景初始图像，学习模型输出各物体对完成该任务的相关性。通过将模型预测作为启发式信息整合到TAMP框架中，搜索空间的规模显著缩减。实验结果表明，与最先进的TAMP求解器相比，我们的框架能够更高效地找到可行解。