Embodied artificial intelligence (EAI) integrates advanced AI models into physical entities for real-world interaction. The emergence of foundation models as the "brain" of EAI agents for high-level task planning has shown promising results. However, the deployment of these agents in physical environments presents significant safety challenges. For instance, a housekeeping robot lacking sufficient risk awareness might place a metal container in a microwave, potentially causing a fire. To address these critical safety concerns, comprehensive pre-deployment risk assessments are imperative. This study introduces EAIRiskBench, a novel framework for automated physical risk assessment in EAI scenarios. EAIRiskBench employs a multi-agent cooperative system that leverages various foundation models to generate safety guidelines, create risk-prone scenarios, make task planning, and evaluate safety systematically. Utilizing this framework, we construct EAIRiskDataset, comprising diverse test cases across various domains, encompassing both textual and visual scenarios. Our comprehensive evaluation of state-of-the-art foundation models reveals alarming results: all models exhibit high task risk rates (TRR), with an average of 95.75% across all evaluated models. To address these challenges, we further propose two prompting-based risk mitigation strategies. While these strategies demonstrate some efficacy in reducing TRR, the improvements are limited, still indicating substantial safety concerns. This study provides the first large-scale assessment of physical risk awareness in EAI agents. Our findings underscore the critical need for enhanced safety measures in EAI systems and provide valuable insights for future research directions in developing safer embodied artificial intelligence system.

翻译：具身人工智能（EAI）将先进的人工智能模型集成到物理实体中，以实现与现实世界的交互。以基础模型作为EAI智能体进行高层任务规划的“大脑”已展现出显著成效。然而，这些智能体在物理环境中的部署带来了严峻的安全挑战。例如，一个缺乏充分风险意识的家务机器人可能将金属容器放入微波炉，从而引发火灾。为解决这些关键安全问题，全面的部署前风险评估势在必行。本研究提出了EAIRiskBench，一个用于EAI场景中自动化物理风险评估的新型框架。EAIRiskBench采用多智能体协同系统，利用多种基础模型来生成安全准则、创建高风险场景、制定任务规划并进行系统性安全评估。基于该框架，我们构建了EAIRiskDataset，包含跨多个领域的多样化测试用例，涵盖文本与视觉场景。我们对前沿基础模型的综合评估揭示了令人警觉的结果：所有模型均表现出较高的任务风险率（TRR），所有评估模型的平均TRR高达95.75%。为应对这些挑战，我们进一步提出了两种基于提示的风险缓解策略。虽然这些策略在降低TRR方面显示出一定效果，但改进有限，仍表明存在重大安全隐患。本研究首次对EAI智能体的物理风险意识进行了大规模评估。我们的发现强调了增强EAI系统安全措施的迫切性，并为未来开发更安全的具身人工智能系统提供了重要的研究方向参考。