Recent advancements in large language models (LLMs) have enabled a new research domain, LLM agents, for solving robotics and planning tasks by leveraging the world knowledge and general reasoning abilities of LLMs obtained during pretraining. However, while considerable effort has been made to teach the robot the "dos," the "don'ts" received relatively less attention. We argue that, for any practical usage, it is as crucial to teach the robot the "don'ts": conveying explicit instructions about prohibited actions, assessing the robot's comprehension of these restrictions, and, most importantly, ensuring compliance. Moreover, verifiable safe operation is essential for deployments that satisfy worldwide standards such as ISO 61508, which defines standards for safely deploying robots in industrial factory environments worldwide. Aiming at deploying the LLM agents in a collaborative environment, we propose a queryable safety constraint module based on linear temporal logic (LTL) that simultaneously enables natural language (NL) to temporal constraints encoding, safety violation reasoning and explaining, and unsafe action pruning. To demonstrate the effectiveness of our system, we conducted experiments in VirtualHome environment and on a real robot. The experimental results show that our system strictly adheres to the safety constraints and scales well with complex safety constraints, highlighting its potential for practical utility.

翻译：近年来，大语言模型（LLM）的进步催生了LLM智能体这一全新研究方向，通过利用预训练期间获取的世界知识与通用推理能力，解决机器人及规划任务。然而，尽管大量研究致力于教导机器人“该做什么”，关于“不该做什么”的关注相对不足。我们认为，对于任何实际应用而言，教导机器人“不该做什么”同样关键：即明确传达禁止行为的指令、评估机器人对这些限制的理解程度，且最重要的是确保其遵守。此外，可验证的安全操作对于满足全球标准（如ISO 61508——该标准定义了在全球工业工厂环境中安全部署机器人的规范）的部署至关重要。为在协作环境中部署LLM智能体，我们提出一种基于线性时序逻辑（LTL）的可查询安全约束模块，该模块同时支持自然语言到时序约束的编码、安全违规的推理与解释，以及不安全动作的剪枝。为验证系统的有效性，我们在VirtualHome环境及真实机器人上进行了实验。结果表明，本系统严格遵循安全约束，并能良好适应复杂的安全约束，凸显了其实用潜力。