In this paper, we introduce a novel fine-tuning technique for language models, which involves incorporating symmetric noise into the embedding process. This method aims to enhance the model's function by more stringently regulating its local curvature, demonstrating superior performance over the current method, NEFTune. When fine-tuning the LLaMA-2-7B model using Alpaca, standard techniques yield a 29.79% score on AlpacaEval. However, our approach, SymNoise, increases this score significantly to 69.04%, using symmetric noisy embeddings. This is a 6.7% improvement over the state-of-the-art method, NEFTune~(64.69%). Furthermore, when tested on various models and stronger baseline instruction datasets, such as Evol-Instruct, ShareGPT, OpenPlatypus, SymNoise consistently outperforms NEFTune. The current literature, including NEFTune, has underscored the importance of more in-depth research into the application of noise-based strategies in the fine-tuning of language models. Our approach, SymNoise, is another significant step towards this direction, showing notable improvement over the existing state-of-the-art method.

翻译：本文提出了一种新颖的语言模型微调技术，该方法通过将对称噪声引入嵌入过程来实现。该技术旨在通过更严格地约束模型局部曲率来增强模型功能，其性能显著优于现有方法NEFTune。在基于Alpaca数据集对LLaMA-2-7B模型进行微调时，标准技术仅获得29.79%的AlpacaEval评分，而本文提出的SymNoise方法通过使用对称噪声嵌入将评分显著提升至69.04%，相较现有最先进的NEFTune方法（64.69%）实现了6.7%的性能提升。此外，在各类模型及更强基准指令数据集（如Evol-Instruct、ShareGPT、OpenPlatypus）上的测试表明，SymNoise方法始终优于NEFTune。现有文献（包括NEFTune）已凸显了噪声策略在语言模型微调应用中进行更深入研究的必要性。本文提出的SymNoise方法正是朝此方向迈出的重要一步，较现有最先进方法展现出显著改进。