Large language models (LLMs) have shown strong capabilities in multi-step decision-making, planning and actions, and are increasingly integrated into various real-world applications. It is concerning whether their strong problem-solving abilities may be misused for crimes. To address this gap, we propose VirtualCrime, a sandbox simulation framework based on a three-agent system to evaluate the criminal capabilities of models. Specifically, this framework consists of an attacker agent acting as the leader of a criminal team, a judge agent determining the outcome of each action, and a world manager agent updating the environment state and entities. Furthermore, we design 40 diverse crime tasks within this framework, covering 11 maps and 13 crime objectives such as theft, robbery, kidnapping, and riot. We also introduce a human player baseline for reference to better interpret the performance of LLM agents. We evaluate 8 strong LLMs and find (1) All agents in the simulation environment compliantly generate detailed plans and execute intelligent crime processes, with some achieving relatively high success rates; (2) In some cases, agents take severe action that inflicts harm to NPCs to achieve their goals. Our work highlights the need for safety alignment when deploying agentic AI in real-world settings.

翻译：大型语言模型（LLMs）在分步决策、规划与行动方面展现出强大能力，并日益融入各类现实应用。其强大的问题解决能力是否可能被滥用于犯罪活动，这一问题令人担忧。为填补这一研究空白，我们提出VirtualCrime——一个基于三智能体系统的沙盒仿真框架，用于评估模型的犯罪能力。具体而言，该框架包含：作为犯罪团队领导者的攻击者智能体、判定每个行动结果的法官智能体，以及更新环境状态与实体的世界管理智能体。此外，我们在此框架内设计了40项多样化的犯罪任务，涵盖11张地图及盗窃、抢劫、绑架、暴乱等13类犯罪目标。我们还引入了人类玩家基线作为参照，以更好地解读LLM智能体的表现。通过对8个主流大型语言模型的评估，我们发现：（1）仿真环境中的所有智能体均能生成详细计划并执行智能化的犯罪过程，部分智能体取得了较高的成功率；（2）在某些情况下，智能体会采取对非玩家角色造成伤害的极端行动以实现其目标。本研究强调了在现实场景中部署具身智能体时进行安全对齐的必要性。