Anticipating and adapting to failures is a key capability robots need to collaborate effectively with humans in complex domains. This continues to be a challenge despite the impressive performance of state of the art AI planning systems and Large Language Models (LLMs) because of the uncertainty associated with the tasks and their outcomes. Toward addressing this challenge, we present a hybrid framework that integrates the generic prediction capabilities of an LLM with the probabilistic sequential decision-making capability of Relational Dynamic Influence Diagram Language. For any given task, the robot reasons about the task and the capabilities of the human attempting to complete it; predicts potential failures due to lack of ability (in the human) or lack of relevant domain objects; and executes actions to prevent such failures or recover from them. Experimental evaluation in the VirtualHome 3D simulation environment demonstrates substantial improvement in performance compared with state of the art baselines.

翻译：在复杂领域中，机器人需要具备预见和适应故障的能力，以与人类进行有效协作。尽管当前最先进的人工智能规划系统和大型语言模型（LLMs）表现出色，但由于任务及其结果存在不确定性，这仍然是一个挑战。为应对这一挑战，我们提出了一种混合框架，该框架将LLM的通用预测能力与关系动态影响图语言的概率序贯决策能力相结合。对于任何给定任务，机器人会推理任务本身及试图完成该任务的人类的能力；预测由于人类能力不足或相关领域对象缺失而可能导致的故障；并执行行动以防止此类故障或从中恢复。在VirtualHome 3D仿真环境中的实验评估表明，与现有最先进的基线方法相比，该框架在性能上取得了显著提升。