Large Language Models (LLMs) have seen remarkable advancements, achieving state-of-the-art results in diverse applications. Fine-tuning, an important step for adapting LLMs to specific downstream tasks, typically involves further training on corresponding datasets. However, a fundamental discrepancy exists between current fine-tuning datasets and the token-level optimization mechanism of LLMs: most datasets are designed at the sentence-level, which introduces token-level noise, causing negative influence to final performance. In this paper, we propose XTF, an explainable token-level noise filtering framework. XTF decomposes the complex and subtle contributions of token-level data to the fine-tuning process into three distinct and explicit attributes (reasoning importance, knowledge novelty, and task relevance), which can be assessed using scoring methods, and then masks the gradients of selected noisy tokens accordingly to optimize the performance of fine-tuned LLMs. We conduct extensive experiments on three representative downstream tasks (math, code and medicine) across 7 mainstream LLMs. The results demonstrate that XTF can significantly improve downstream performance by up to 13.7% compared to regular fine-tuning. Our work highlights the importance of token-level dataset optimization, and demonstrates the potential of strategies based on attribute decomposition for explaining complex training mechanisms.

翻译：大语言模型（LLMs）已取得显著进展，在多样化应用中实现了最先进的性能。微调作为将LLMs适配至特定下游任务的关键步骤，通常需要在相应数据集上进行进一步训练。然而，当前微调数据集与LLMs的词元级优化机制之间存在根本性差异：大多数数据集在句子层面进行设计，这引入了词元级噪声，对最终性能产生负面影响。本文提出XTF，一种可解释的词元级噪声过滤框架。XTF将词元级数据对微调过程复杂而细微的贡献分解为三个独立且明确的属性（推理重要性、知识新颖性和任务相关性），这些属性可通过评分方法进行评估，随后据此对选定噪声词元的梯度进行掩码，以优化微调后LLMs的性能。我们在三个代表性下游任务（数学、代码和医学）上对7种主流LLM进行了广泛实验。结果表明，与常规微调相比，XTF最高可将下游性能提升13.7%。本研究强调了词元级数据集优化的重要性，并展示了基于属性分解的策略在解释复杂训练机制方面的潜力。