In this paper, we introduce a novel fine-tuning technique for language models, which involves incorporating symmetric noise into the embedding process. This method aims to enhance the model's function by more stringently regulating its local curvature, demonstrating superior performance over the current method, NEFTune. When fine-tuning the LLaMA-2-7B model using Alpaca, standard techniques yield a 29.79% score on AlpacaEval. However, our approach, SymNoise, increases this score significantly to 69.04%, using symmetric noisy embeddings. This is a 6.7% improvement over the state-of-the-art method, NEFTune~(64.69%). Furthermore, when tested on various models and stronger baseline instruction datasets, such as Evol-Instruct, ShareGPT, OpenPlatypus, SymNoise consistently outperforms NEFTune. The current literature, including NEFTune, has underscored the importance of more in-depth research into the application of noise-based strategies in the fine-tuning of language models. Our approach, SymNoise, is another significant step towards this direction, showing notable improvement over the existing state-of-the-art method.

翻译：本文提出了一种新型语言模型微调技术，该方法在嵌入过程中引入对称噪声。该技术通过更严格地约束模型局部曲率来增强模型功能，其性能显著优于现有方法NEFTune。在使用Alpaca数据集微调LLaMA-2-7B模型时，标准技术仅获得29.79%的AlpacaEval评分。而本文提出的SymNoise方法通过对称噪声嵌入将该分数大幅提升至69.04%，相比当前最先进的NEFTune方法（64.69%）提升了6.7%。进一步在多种模型及更强基线指令数据集（如Evol-Instruct、ShareGPT、OpenPlatypus）上的测试表明，SymNoise始终优于NEFTune。现有文献（包括NEFTune）已强调需要对噪声策略在语言模型微调中的应用进行更深入的研究。本文提出的SymNoise方法正是向该方向迈出的重要一步，较当前最先进方法实现了显著改进。