We introduce SenseAI, a human-in-the-loop (HITL) validated financial sentiment dataset designed to capture not only model outputs but the full reasoning process behind them. Unlike existing resources, SenseAI incorporates reasoning chains, confidence scores, human correction signals, and real-world market outcomes, providing a structure aligned with Reinforcement Learning from Human Feedback (RLHF) paradigms. The dataset consists of 1,439 labelled data points across 40 US-listed equities and 13 financial data categories, enabling direct integration into modern LLM fine-tuning pipelines. Through analysis, we identify several systematic patterns in model behavior, including a novel failure mode we term Latent Reasoning Drift, where models introduce information not grounded in the input, as well as consistent confidence miscalibration and forward projection tendencies. These findings suggest that LLM errors in financial reasoning are not random but occur within a predictable and correctable regime, supporting the use of structured HITL data for targeted model improvement. We discuss implications for financial AI systems and highlight opportunities for applying SenseAI in model evaluation and alignment.

翻译：摘要：我们提出SenseAI——一个经人机交互验证的金融情感数据集，其独特之处在于不仅记录模型输出，更完整捕捉输出背后的推理过程。与现有资源不同，该数据集整合了推理链、置信度分数、人工修正信号及真实市场结果，构建出与基于人类反馈的强化学习范式相兼容的数据结构。数据集涵盖40只美国上市股票、13个金融数据类别共1439个标注数据点，可直接集成到现代大语言模型微调流程中。通过分析，我们识别出模型行为中的若干系统性模式，包括被称为潜在推理偏移的新型失效模式（模型引入非输入信息）、一致的置信度校准偏差及前向投影倾向。这些发现表明大语言模型在金融推理中的错误并非随机，而是出现在可预测且可修正的范围内，从而支持使用结构化人机交互数据进行针对性模型改进。本文探讨了对金融AI系统的启示，并指出了SenseAI在模型评估与对齐领域的应用前景。