Large language models (LLMs) frequently produce contextual hallucinations, where generated content contradicts or ignores information explicitly stated in the prompt. Such errors are particularly problematic in deterministic automation workflows, where inputs are fixed and correctness is unambiguous. We introduce a simple and model-agnostic framework that provides explicit probabilistic guarantees for reducing hallucinations in this setting. We formalize the notion of a specific task, defined by a fixed input and a deterministic correctness criterion, and show that issuing the same prompt in independent context windows yields an exponential reduction in the probability that all model outputs are incorrect. To identify a correct answer among repeated runs, we incorporate an LLM-as-a-judge and prove that the probability that the judged pipeline fails decays at a rate determined by the judge's true- and false-positive probabilities. When the judge is imperfect, we strengthen it through majority vote over independent judge calls, obtaining ensemble-level error rates that decrease exponentially in the number of votes. This yields an explicit bound on the probability that the pipeline selects a hallucinated answer. Experiments on controlled extraction tasks with synthetic noisy judges match these predictions exactly: pipeline failure decreases exponentially with the number of repetitions, and hallucination-selection decreases exponentially with the number of judges in the ensemble. Together, these results provide a lightweight, modular, and theoretically grounded method for driving hallucination probabilities arbitrarily low in fixed-input LLM workflows-without modifying model weights, decoding strategies, or prompt engineering.
翻译:大型语言模型(LLMs)经常产生上下文幻觉,即生成的内容与提示中明确陈述的信息相矛盾或忽略这些信息。此类错误在确定性自动化工作流中尤其成问题,因为其输入是固定的且正确性标准明确。我们提出了一种简单且与模型无关的框架,为在此类场景中减少幻觉提供了明确的概率保证。我们形式化了特定任务的概念,该任务由固定输入和确定性正确性标准定义,并证明在独立的上下文窗口中重复发出相同提示,能够使所有模型输出均不正确的概率呈指数级下降。为了在多次运行中识别正确答案,我们引入了LLM-as-a-judge机制,并证明了该判定流程失败的概率以判定器真阳性率和假阳性率为决定因素进行衰减。当判定器不完美时,我们通过对其独立判定结果进行多数投票来增强其性能,从而获得随投票数增加呈指数下降的集成级错误率。这为流程选择幻觉答案的概率提供了明确上界。在受控信息提取任务上使用合成噪声判定器进行的实验完全验证了这些预测:流程失败率随重复次数增加呈指数下降,幻觉选择率随集成中判定器数量增加呈指数下降。综上,这些结果为在固定输入的LLM工作流中(无需修改模型权重、解码策略或提示工程)将幻觉概率任意降低,提供了一种轻量级、模块化且理论依据充分的方法。