Contemporary large language models (LLMs) have demonstrated remarkable reasoning capabilities, particularly in specialized domains like mathematics and physics. However, their ability to generalize these reasoning skills to more general and broader contexts--often termed general reasoning--remains under-explored. Unlike domain-specific reasoning, general reasoning relies less on expert knowledge but still presents formidable reasoning challenges, such as complex constraints, nested logical branches, and semantic interference. To address this gap, we introduce General365, a benchmark specifically designed to assess general reasoning in LLMs. By restricting background knowledge to a K-12 level, General365 explicitly decouples reasoning from specialized expertise. The benchmark comprises 365 seed problems and 1,095 variant problems across eight categories, ensuring both high difficulty and diversity. Evaluations across 26 leading LLMs reveal that even the top-performing model achieves only 62.8% accuracy, in stark contrast to the near-perfect performances of LLMs in math and physics benchmarks. These results suggest that the reasoning abilities of current LLMs are heavily domain-dependent, leaving significant room for improvement in broader applications. We envision General365 as a catalyst for advancing LLM reasoning beyond domain-specific tasks toward robust, general-purpose real-world scenarios. Code, Dataset, and Leaderboard: https://general365.github.io
翻译:当代大语言模型(LLM)已在数学、物理等专业领域展现出卓越的推理能力。然而,将这些推理技能泛化至更广泛通用场景(即通用推理)的能力仍鲜有探究。与领域特定推理不同,通用推理较少依赖专业知识,但依然面临复杂约束条件、嵌套逻辑分支及语义干扰等严峻推理挑战。为弥补这一研究空白,我们提出通用推理基准测试General365,通过将背景知识限制在K-12教育水平,明确剥离推理与专业知识的关联。该基准包含365道种子题与1095道变式题,涵盖八大类别,兼具高难度与多样性。在26款主流大语言模型上的评估显示,即便表现最优的模型准确率也仅为62.8%,与现有数学、物理基准测试中近乎完美的表现形成鲜明对比。结果表明,当前大语言模型推理能力存在严重的领域依赖性,在更广泛的应用场景中存在显著提升空间。我们期望General365能推动大语言模型推理从领域特定任务向稳健的通用真实场景发展。代码、数据与排行榜:https://general365.github.io