Although large language models (LLMs) have demonstrated impressive text generation capabilities, they are easily misled by the untruthful context provided by users or knowledge argumentation tools, thereby producing hallucinations. To alleviate the LLMs from being misled by untruthful information and take advantage of knowledge argumentation, we propose Truth-Aware Context Selection (TACS), a lightweight method to shield untruthful context from the inputs. TACS begins by performing truth detection on the input context, leveraging the parameterized knowledge within the LLM. Subsequently, it constructs a corresponding attention mask based on the truthfulness of each position, selecting the truthful context and discarding the untruthful context. Additionally, we introduce a new evaluation metric, Disturbance Adaption Rate, to further study the LLMs' ability to accept truthful information and resist untruthful information. Experimental results show that TACS can effectively filter information in context and significantly improve the overall quality of LLMs' responses when presented with misleading information.

翻译：尽管大语言模型（LLMs）已展现出令人印象深刻的文本生成能力，但它们极易被用户或知识论证工具提供的不真实上下文所误导，从而产生幻觉。为缓解LLMs被不真实信息误导的问题，同时发挥知识论证的优势，我们提出真实感知上下文选择（TACS）——一种轻量级方法，用于从输入中屏蔽不真实上下文。TACS首先利用LLM内部的参数化知识对输入上下文进行真实性检测；随后，根据每个位置的真实性构建相应的注意力掩码，选择真实上下文并丢弃不真实上下文。此外，我们引入了一种新的评估指标——干扰适应率，以进一步研究LLMs接受真实信息和抵抗不真实信息的能力。实验结果表明，当面对误导信息时，TACS能够有效过滤上下文中的信息，显著提升LLMs响应的整体质量。