Recent advancements in large language models (LLMs) have enabled a new research domain, LLM agents, for solving robotics and planning tasks by leveraging the world knowledge and general reasoning abilities of LLMs obtained during pretraining. However, while considerable effort has been made to teach the robot the "dos," the "don'ts" received relatively less attention. We argue that, for any practical usage, it is as crucial to teach the robot the "don'ts": conveying explicit instructions about prohibited actions, assessing the robot's comprehension of these restrictions, and, most importantly, ensuring compliance. Moreover, verifiable safe operation is essential for deployments that satisfy worldwide standards such as ISO 61508, which defines standards for safely deploying robots in industrial factory environments worldwide. Aiming at deploying the LLM agents in a collaborative environment, we propose a queryable safety constraint module based on linear temporal logic (LTL) that simultaneously enables natural language (NL) to temporal constraints encoding, safety violation reasoning and explaining, and unsafe action pruning. To demonstrate the effectiveness of our system, we conducted experiments in VirtualHome environment and on a real robot. The experimental results show that our system strictly adheres to the safety constraints and scales well with complex safety constraints, highlighting its potential for practical utility.

翻译：大型语言模型（LLM）的最新进展催生了一个新的研究领域——LLM智能体，它利用预训练过程中LLM获得的世界知识和通用推理能力来解决机器人与规划任务。然而，尽管大量工作致力于教导机器人“该做什么”，关于“不该做什么”的关注却相对较少。我们认为，在任何实际应用中，教导机器人“不该做什么”同样至关重要：传达关于禁止行为的明确指令、评估机器人对这些限制的理解，以及最重要的是确保其遵守。此外，可验证的安全运行对于满足全球标准（例如ISO 61508，该标准定义了在工业工厂环境中安全部署机器人的规范）的部署至关重要。为了在协作环境中部署LLM智能体，我们提出了一种基于线性时序逻辑（LTL）的可查询安全约束模块，该模块同时支持自然语言（NL）到时序约束编码、安全违规推理与解释，以及不安全动作剪枝。为了证明我们系统的有效性，我们在VirtualHome环境和真实机器人上进行了实验。实验结果表明，我们的系统严格遵守安全约束，并能良好应对复杂安全约束的扩展，凸显了其实际应用潜力。