Large Language Models (LLMs) have seen remarkable advancements, achieving state-of-the-art results in diverse applications. Fine-tuning, an important step for adapting LLMs to specific downstream tasks, typically involves further training on corresponding datasets. However, a fundamental discrepancy exists between current fine-tuning datasets and the token-level optimization mechanism of LLMs: most datasets are designed at the sentence-level, which introduces token-level noise, causing negative influence to final performance. In this paper, we propose XTF, an explainable token-level noise filtering framework. XTF decomposes the complex and subtle contributions of token-level data to the fine-tuning process into three distinct and explicit attributes (reasoning importance, knowledge novelty, and task relevance), which can be assessed using scoring methods, and then masks the gradients of selected noisy tokens accordingly to optimize the performance of fine-tuned LLMs. We conduct extensive experiments on three representative downstream tasks (math, code and medicine) across 7 mainstream LLMs. The results demonstrate that XTF can significantly improve downstream performance by up to 13.7% compared to regular fine-tuning. Our work highlights the importance of token-level dataset optimization, and demonstrates the potential of strategies based on attribute decomposition for explaining complex training mechanisms.

翻译：大语言模型（LLMs）在各类应用场景中取得了显著进展，达到了最先进的性能水平。微调作为将LLMs适配到特定下游任务的关键步骤，通常需要基于相应数据集进行进一步训练。然而，当前微调数据集与LLMs的词元级优化机制存在根本性矛盾：大部分数据集采用句子级设计，这引入了词元级噪声，对最终性能产生负面影响。本文提出XTF——一种可解释的词元级噪声过滤框架。该框架将词元级数据对微调过程的复杂细微贡献分解为三个明确可辨的属性（推理重要性、知识新颖性和任务相关性），并通过评分方法进行评估，进而对选定噪声词元的梯度进行掩码处理，以优化微调LLMs的性能。我们在数学、代码和医学三个代表性下游任务上，针对7种主流LLMs进行了大量实验。结果表明，相较于常规微调，XTF可将下游性能最高提升13.7%。我们的工作凸显了词元级数据集优化的重要性，并展示了基于属性分解策略在解释复杂训练机制方面的潜力。