LLMs and VLMs are increasingly deployed as embodied agents, yet existing benchmarks largely revolve around simple short-term tasks and struggle to capture rich realistic constraints that shape real-world decision making. To close this gap, we propose DeliveryBench, a city-scale embodied benchmark grounded in the real-world profession of food delivery. Food couriers naturally operate under long-horizon objectives (maximizing net profit over hours) while managing diverse constraints, e.g., delivery deadline, transportation expense, vehicle battery, and necessary interactions with other couriers and customers. DeliveryBench instantiates this setting in procedurally generated 3D cities with diverse road networks, buildings, functional locations, transportation modes, and realistic resource dynamics, enabling systematic evaluation of constraint-aware, long-horizon planning. We benchmark a range of VLM-based agents across nine cities and compare them with human players. Our results reveal a substantial performance gap to humans, and find that these agents are short-sighted and frequently break basic commonsense constraints. Additionally, we observe distinct personalities across models (e.g., adventurous GPT-5 vs. conservative Claude), highlighting both the brittleness and the diversity of current VLM-based embodied agents in realistic, constraint-dense environments. Our code, data, and benchmark are available at https://deliverybench.github.io.

翻译：大型语言模型（LLM）和视觉语言模型（VLM）正越来越多地被部署为具身智能体，然而现有基准测试大多围绕简单的短期任务展开，难以捕捉塑造现实世界决策的丰富且真实的约束条件。为弥补这一差距，我们提出了DeliveryBench——一个基于现实世界外卖配送职业构建的城市规模具身基准测试。外卖骑手天然地在长时程目标（数小时内最大化净收益）下运作，同时需管理多种约束，例如配送截止时间、交通成本、车辆电量，以及与其他骑手和客户的必要互动。DeliveryBench在程序化生成的三维城市中实例化了这一场景，这些城市拥有多样化的道路网络、建筑、功能地点、交通模式以及真实的资源动态，从而能够系统性地评估具有约束感知能力的长时程规划。我们在九个城市中对一系列基于VLM的智能体进行了基准测试，并将其与人类玩家进行了比较。我们的结果揭示了这些智能体与人类之间存在显著的性能差距，并发现它们往往目光短浅，且频繁违反基本的常识性约束。此外，我们观察到不同模型展现出鲜明的个性差异（例如，富有冒险精神的GPT-5与保守的Claude），这突显了当前基于VLM的具身智能体在真实、约束密集环境中的脆弱性与多样性。我们的代码、数据和基准测试可在 https://deliverybench.github.io 获取。