Foundation models are now used in settings where the prompts they receive can change quickly. Users change, topics change, policies change, and the model may suddenly face a kind of request that was rare in the calibration data. This makes fixed calibration risky. Conformal prediction and conformal risk control give model-agnostic ways to control error, but they work best when the calibration data still look like the future data. This paper develops PromptShift CRC, a drift-aware conformal risk control method for foundation-model outputs under prompt and domain shift. The method embeds prompts and responses, measures how far the current prompt stream has moved from the calibration pool, gives more weight to relevant or recent calibration examples, and updates the risk level online after observed violations. It reports three practical diagnostics: realized risk error, prompt drift, and effective calibration size. We give conditions under which the method controls risk up to terms for distribution mismatch and weighted quantile uncertainty. In a synthetic prompt-shift benchmark, static conformal risk control fails sharply after drift, while PromptShift-CRC gives the best coverage among the adaptive baselines considered. We then evaluate the same calibration layer on public benchmark derived streams for question answering, toxicity, summarization factuality, and long-context hallucination risk
翻译:基础模型现已应用于其接收到的提示可能快速变化的场景中。用户会变化、话题会变化、策略会变化,模型可能突然面临在校准数据中罕见的请求类型,这使得固定校准存在风险。保形预测与保形风险控制提供了与模型无关的错误控制方法,但它们在校准数据与未来数据特征相似时表现最佳。本文提出了PromptShift-CRC,一种面向基础模型输出在提示与领域漂移下的漂移感知保形风险控制方法。该方法将提示与响应嵌入,度量当前提示流相较于校准池的偏移程度,为相关或近期校准样本赋予更高权重,并在观测到违规后在线更新风险水平。我们报告了三个实用诊断指标:实际风险误差、提示漂移与有效校准规模。我们给出了该方法在分布不匹配与加权分位数不确定性项上控制风险的条件。在合成提示漂移基准测试中,静态保形风险控制在漂移后显著失效,而PromptShift-CRC在考虑的适应性基线方法中达到了最佳覆盖范围。随后,我们在面向问答、毒性检测、摘要事实性及长上下文幻觉风险的公开基准衍生流上评估了相同的校准层。