Large Language Models (LLMs) have become proficient in addressing complex tasks by leveraging their extensive internal knowledge and reasoning capabilities. However, the black-box nature of these models complicates the task of explaining their decision-making processes. While recent advancements demonstrate the potential of leveraging LLMs to self-explain their predictions through natural language (NL) explanations, their explanations may not accurately reflect the LLMs' decision-making process due to a lack of fidelity optimization on the derived explanations. Measuring the fidelity of NL explanations is a challenging issue, as it is difficult to manipulate the input context to mask the semantics of these explanations. To this end, we introduce FaithLM to explain the decision of LLMs with NL explanations. Specifically, FaithLM designs a method for evaluating the fidelity of NL explanations by incorporating the contrary explanations to the query process. Moreover, FaithLM conducts an iterative process to improve the fidelity of derived explanations. Experiment results on three datasets from multiple domains demonstrate that FaithLM can significantly improve the fidelity of derived explanations, which also provides a better alignment with the ground-truth explanations.

翻译：大型语言模型（LLMs）通过利用其丰富的内部知识和推理能力，已能熟练处理复杂任务。然而，这些模型的黑箱特性使得解释其决策过程变得复杂。尽管近期研究进展表明，利用LLMs通过自然语言（NL）解释进行自我预测具有潜力，但由于缺乏对衍生解释的忠实度优化，这些解释可能无法准确反映LLMs的决策过程。衡量NL解释的忠实度是一个具有挑战性的问题，因为难以通过操纵输入上下文来掩盖这些解释的语义。为此，我们提出FaithLM，旨在用NL解释来阐明LLMs的决策。具体而言，FaithLM设计了一种通过将对立解释纳入查询过程来评估NL解释忠实度的方法。此外，FaithLM通过迭代过程提升衍生解释的忠实度。在多个领域三个数据集上的实验结果表明，FaithLM能显著提高衍生解释的忠实度，同时与真实解释实现更好的对齐。