Recent advancements in large language models (LLMs) have enabled a new research domain, LLM agents, for solving robotics and planning tasks by leveraging the world knowledge and general reasoning abilities of LLMs obtained during pretraining. However, while considerable effort has been made to teach the robot the "dos," the "don'ts" received relatively less attention. We argue that, for any practical usage, it is as crucial to teach the robot the "don'ts": conveying explicit instructions about prohibited actions, assessing the robot's comprehension of these restrictions, and, most importantly, ensuring compliance. Moreover, verifiable safe operation is essential for deployments that satisfy worldwide standards such as ISO 61508, which defines standards for safely deploying robots in industrial factory environments worldwide. Aiming at deploying the LLM agents in a collaborative environment, we propose a queryable safety constraint module based on linear temporal logic (LTL) that simultaneously enables natural language (NL) to temporal constraints encoding, safety violation reasoning and explaining, and unsafe action pruning. To demonstrate the effectiveness of our system, we conducted experiments in VirtualHome environment and on a real robot. The experimental results show that our system strictly adheres to the safety constraints and scales well with complex safety constraints, highlighting its potential for practical utility.

翻译：近年来，大型语言模型（LLM）的进展催生了一个新的研究领域——LLM智能体，通过利用预训练中获得的世界知识与通用推理能力，可解决机器人与规划任务。然而，尽管大量研究致力于教导机器人“该做什么”，“不该做什么”却受到的关注相对较少。我们认为，在任何实际应用中，教导机器人“不该做什么”同样至关重要：明确传达禁止执行的指令、评估机器人的理解程度，以及最重要的是确保其合规性。此外，可验证的安全操作对于符合ISO 61508等全球标准的部署至关重要——该标准定义了全球工业厂房环境中机器人安全部署的规范。旨在将LLM智能体部署于协作环境，我们提出一种基于线性时态逻辑（LTL）的可查询安全约束模块，该模块能同时实现自然语言（NL）到时态约束的编码、安全违规推理与解释，以及不安全动作剪枝。为验证系统有效性，我们在VirtualHome环境和真实机器人上开展实验。结果表明，该系统严格遵循安全约束，并能良好扩展至复杂安全场景，凸显其实用潜力。