Travel planning stands out among real-world applications of \emph{Language Agents} because it couples significant practical demand with a rigorous constraint-satisfaction challenge. However, existing benchmarks primarily operate on a slot-filling paradigm, restricting agents to synthetic queries with pre-defined constraint menus, which fails to capture the open-ended nature of natural language interaction, where user requirements are compositional, diverse, and often implicitly expressed. To address this gap, we introduce \emph{ChinaTravel}, with four key contributions: 1) a practical sandbox aligned with the multi-day, multi-POI travel planning, 2) a compositionally generalizable domain-specific language (DSL) for scalable evaluation, covering feasibility, constraint satisfaction, and preference comparison 3) an open-ended dataset that integrates diverse travel requirements and implicit intent from 1154 human participants, and 4) fine-grained analysis reveal the potential of neuro-symbolic agents in travel planning, achieving a 37.0% constraint satisfaction rate on human queries, a 10 \times improvement over purely neural models, yet highlighting significant challenges in compositional generalization. Overall, ChinaTravel provides a foundation for advancing language agents through compositional constraint validation in complex, real-world planning scenarios. Project Page: https://www.lamda.nju.edu.cn/shaojj/ChinaTravel/index.html
翻译:旅行规划在语言智能体的实际应用中脱颖而出,因为它既承载着显著的实际需求,又伴随着严格的约束满足挑战。然而,现有基准测试主要基于填槽范式运行,将智能体限制在预定义约束菜单的合成查询中,未能捕捉自然语言交互的开放性本质——用户需求具有组合性、多样性,且常以隐含方式表达。为填补这一空白,我们提出中国旅行(ChinaTravel)基准,贡献包括:1)一个与多日、多兴趣点旅行规划场景对齐的实用沙盒环境;2)一种可组合泛化的领域特定语言(DSL),用于可扩展评估(涵盖可行性、约束满足与偏好比较);3)包含1154名人类参与者多样化旅行需求与隐含意图的开放式数据集;4)细粒度分析揭示了神经符号智能体在旅行规划中的潜力——对人类查询实现了37.0%的约束满足率,是纯神经模型的10倍提升,同时暴露出组合泛化方面的重大挑战。总体而言,中国旅行通过复杂现实规划场景中的组合约束验证,为推进语言智能体研究提供了基础。项目主页:https://www.lamda.nju.edu.cn/shaojj/ChinaTravel/index.html