Large Language Models (LLMs) encode vast amounts of parametric knowledge during pre-training. As world knowledge evolves, effective deployment increasingly depends on their ability to faithfully follow externally retrieved context. When such evidence conflicts with the model's internal knowledge, LLMs often default to memorized facts, producing unfaithful outputs. In this work, we introduce ContextFocus, a lightweight activation steering approach that improves context faithfulness in such knowledge-conflict settings while preserving fluency and efficiency. Unlike prior approaches, our solution requires no model finetuning and incurs minimal inference-time overhead, making it highly efficient. We evaluate ContextFocus on the ConFiQA benchmark, comparing it against strong baselines including ContextDPO, COIECD, and prompting-based methods. Furthermore, we show that our method is complementary to prompting strategies and remains effective on larger models. Extensive experiments show that ContextFocus significantly improves contextual-faithfulness. Our results highlight the effectiveness, robustness, and efficiency of ContextFocus in improving contextual-faithfulness of LLM outputs.

翻译：大语言模型在预训练过程中编码了大量参数化知识。随着世界知识的不断演进，其有效部署日益依赖于模型忠实遵循外部检索上下文的能力。当此类证据与模型内部知识发生冲突时，大语言模型往往默认采用记忆中的事实，从而产生不忠实的输出。本研究提出ContextFocus——一种轻量级激活导向方法，在保持流畅性与效率的同时，显著提升模型在知识冲突场景下的上下文忠实性。与现有方法不同，本方案无需模型微调且推理时开销极低，具有高度效率优势。我们在ConFiQA基准上评估ContextFocus，并与ContextDPO、COIECD及基于提示的强基线方法进行对比。进一步研究表明，本方法与提示策略具有互补性，且在更大规模模型上仍保持有效性。大量实验证明ContextFocus能显著提升上下文忠实性。研究结果凸显了ContextFocus在改进大语言模型输出上下文忠实性方面的有效性、鲁棒性与高效性。