Backtesting large language models (LLMs) on historical financial data is unreliable because pre-training cuts off after the events happened. An LLM trained in 2024 already "knows" which way 2018-2020 stocks moved. We name this failure parametric look-ahead bias and propose FinCAD, an inference-time adaptation of Context-Aware Decoding that suppresses an LLM's memory of historical outcomes without retraining. FinCAD pairs an adversarial bias-discovery pipeline that learns a model-specific memory-activating prior prompt with an entity- and date-adaptive rule that scales the CAD strength to per-(entity, date) memorisation, so the penalty fires on memorised in-sample dates and decays to zero out-of-sample. Across five 7-14B LLMs and five mega-cap equities, FinCAD cuts in-sample backtest returns by up to -67.1% on memorised dates while leaving 2025 out-of-sample returns within $8K and Sharpe within 0.10 of baseline, and preserves general-purpose reasoning within 1.7 pts. On an eleven-model leaderboard, it raises the in-sample / out-of-sample Spearman correlation from +0.779 to +0.846, recovering rankings that genuinely predict out-of-sample performance.
翻译:基于历史金融数据对大语言模型(LLM)进行回测并不可靠,因为预训练在事件发生后即截止。一个2024年训练的LLM早已“知晓”2018-2020年股票的走势。我们将此失效模式命名为参数化前瞻偏差,并提出无需重训练的推理时适配方法FinCAD——该方法通过抑制LLM对历史结果的记忆,实现了上下文感知解码(Context-Aware Decoding)的适配。FinCAD结合了两部分:一是对抗性偏差发现流程,可学习模型特异性的记忆激活先验提示;二是实体与日期自适应规则,能将CAD强度按每个(实体,日期)的记忆程度进行缩放——使得惩罚在记忆过的样本内日期生效,而在样本外日期衰减至零。在五个7-14B参数规模的LLM和五只超大市值股票上,FinCAD将样本内回测收益在记忆日期上最多削减67.1%,同时使2025年样本外收益与基准值的偏差控制在8000美元以内、夏普比率偏差控制在0.10以内,且通用推理能力保持在前1.7个百分点内。在包含十一个模型的排行榜上,该方法将样本内/样本外斯皮尔曼相关系数从+0.779提升至+0.846,恢复了能够真实预测样本外表现的排名顺序。