As LLM benchmarks saturate, the evaluation community has pursued two strategies to increase difficulty: escalating knowledge demands (GPQA, HLE) or removing knowledge entirely in favor of abstract reasoning (ARC-AGI). The first conflates memorization with capability; the second divorces reasoning from the practical contexts in which it matters. We take a different approach. The Grounded Integration Measure (GIM) is a benchmark of 820 original problems (615 public, 205 private) where difficulty comes from integration; individual problems require coordinating multiple cognitive operations (constraint satisfaction, state tracking, epistemic vigilance, audience calibration) over broadly accessible knowledge, so that reasoning stays grounded in realistic tasks without being gated on specialized expertise. Each problem is an original expert-authored composition, majority with rubric-decomposed scoring (median 6 independently judged criteria). A balanced public--private split provides built-in contamination diagnostic. We calibrate a continuous response 2-parameter logistic (2PL) IRT model over >200k prompt-response pairs across 28 models, producing robust ability estimates that correctly order test-configurations even when raw accuracy is distorted by errors or missing data, addressing a common challenge in benchmark reporting. Using this framework, we present a comprehensive leaderboard spanning 22 models and 47 test-configurations (unique model, thinking-level pairs), and conduct what is to our knowledge the most extensive published study of how test-time compute trades off against model capability on a fixed benchmark: 11 models swept across 35 test-configurations. We observe that within-family configuration choices, such as thinking budget and quantization, matter as much as model selection. We release the evaluation framework, calibrated IRT parameters, and all public problems.

翻译：摘要：随着大语言模型（LLM）基准测试趋于饱和，评估界已采取两种策略来提升难度：一是提高知识需求（如GPQA、HLE），二是完全摒弃知识，转而依赖抽象推理（如ARC-AGI）。前者混淆了记忆与能力，后者则使推理脱离其实践意义所在的现实情境。我们另辟蹊径。基础整合度量（Ground Integration Measure, GIM）是一个包含820道原创问题（615道公开，205道私密）的基准测试，其难度源于整合：每个问题需协调多种认知操作（约束满足、状态追踪、认知警觉、受众校准），并基于广泛可及的知识，使推理扎根于现实任务，而非受限于专业知识的门槛。每道问题均由专家原创编写，多数采用评分细则分解评分（中位数包含6个独立评判标准）。公开与私密问题的平衡划分内置了污染诊断机制。我们基于28个模型的超过20万个提示-响应对，校准了连续响应双参数逻辑斯蒂克（2PL）项目反应理论（IRT）模型，生成了稳健的能力估计值，这些估计值能在原始准确率因误差或缺失数据而失真时，正确地对测试配置进行排序，从而解决了基准报告中的常见难题。利用这一框架，我们推出了涵盖22个模型和47种测试配置（各具特色的模型与思考层级组合）的综合排行榜，并开展了据我们所知关于测试时计算与模型能力在固定基准上如何权衡的最广泛公开研究：11个模型在35种测试配置上进行了扫描。我们观察到，家族内的配置选择（如思考预算和量化）与模型选择同等重要。我们已发布评估框架、校准的IRT参数及所有公开问题。