Existing financial NLP benchmarks often rely on labels supplied by outside observers, measuring how language is perceived rather than what speakers have committed to in the market. We introduce StakeBench, an evaluation framework for language understanding grounded in market commitment. StakeBench links 560,876 comments from 2,261 resolved markets to verified position, action, and market-odds records across Polymarket and Manifold. Supervision is derived from observable market behavior. Position sides, post-comment trading actions, and market-odds trajectories replace human annotation. Four diagnostic tasks test whether models detect market commitment, identify the revealed side, anticipate future action, and perform collective odds projection. Three commitment-aware metrics measure alignment with revealed preferences rather than perceived sentiment. Validity audits and explicit interpretation boundaries help distinguish observable commitment signals from latent belief and causal market-odds impact. Across 15 LLMs and 18 topics and platform settings, models partially recover position-side signals, with Directed Accuracy from 0.506 to 0.599, but show structural failures on later tasks. Ten of the fifteen models collapse to one or two action labels in future action anticipation, and no model consistently improves on the naive odds-direction baseline in collective odds projection. Model scale is not correlated with performance, finance-domain tuning does not improve revealed-side identification, and platform incentives strongly shape higher-order results. StakeBench is packaged with evaluation code and dataset under CC-BY 4.0.

翻译：现有金融自然语言处理基准多依赖外部观察者标注，衡量的是语言被感知的方式而非发言者在市场中所承诺的内容。我们提出StakeBench——一种基于市场承诺的语言理解评估框架。StakeBench将来自2261个已结算市场的560876条评论与Polymarket和Manifold平台上已验证的头寸、交易行为及市场赔率记录相关联。监督信号源自可观测的市场行为：立场方向、评论后交易行为和市场赔率轨迹取代了人工标注。四项诊断任务检验模型是否能够检测市场承诺、识别所揭示的立场方向、预测后续行为以及执行集体赔率预测。三种承诺感知指标衡量模型与所揭示偏好的契合度，而非感知情绪。有效性审计和明确的解释边界有助于区分可观测的承诺信号与潜在信念及因果性市场赔率影响。在15个大型语言模型、18个主题及平台设置中，模型可部分恢复立场方向信号（导向准确率0.506至0.599），但在后续任务中显现结构性缺陷：15个模型中有10个在预测后续行为时崩塌至仅1-2个行为标签，且无任何模型在集体赔率预测中持续优于朴素赔率方向基线。模型规模与性能无关，金融领域微调未能改善揭示立场识别效果，而平台激励机制显著影响高阶结果。StakeBench以CC-BY 4.0协议发布，附带评估代码与数据集。