Robot task planning from high-level instructions is an important step towards deploying fully autonomous robot systems in the service sector. Three key aspects of robot task planning present challenges yet to be resolved simultaneously, namely, (i) factorization of complex tasks specifications into simpler executable subtasks, (ii) understanding of the current task state from raw observations, and (iii) planning and verification of task executions. To address these challenges, we propose LATMOS, an automata-inspired task model that, given observations from correct task executions, is able to factorize the task, while supporting verification and planning operations. LATMOS combines an observation encoder to extract the features from potentially high-dimensional observations with automata theory to learn a sequential model that encapsulates an automaton with symbols in the latent feature space. We conduct extensive evaluations in three task model learning setups: (i) abstract tasks described by logical formulas, (ii) real-world human tasks described by videos and natural language prompts and (iii) a robot task described by image and state observations. The results demonstrate the improved plan generation and verification capabilities of LATMOS across observation modalities and tasks.

翻译：从高层指令进行机器人任务规划是实现服务领域全自主机器人系统部署的关键步骤。机器人任务规划的三个核心方面仍存在尚未同时解决的挑战，即：(i) 将复杂任务规范分解为可执行的简单子任务，(ii) 从原始观测中理解当前任务状态，以及(iii) 任务执行的规划与验证。为解决这些挑战，我们提出LATMOS，一种受自动机启发的任务模型，该模型在给定正确任务执行的观测数据后，能够分解任务，同时支持验证与规划操作。LATMOS结合了观测编码器（用于从潜在高维观测中提取特征）与自动机理论，以学习一个在潜在特征空间中封装了带符号自动机的序列模型。我们在三种任务模型学习场景中进行了广泛评估：(i) 由逻辑公式描述的抽象任务，(ii) 由视频和自然语言提示描述的真实世界人类任务，以及(iii) 由图像和状态观测描述的机器人任务。结果表明，LATMOS在不同观测模态和任务类型中均展现出更强的规划生成与验证能力。