Large Language Models (LLMs) have highlighted the necessity of effective unlearning mechanisms to comply with data regulations and ethical AI practices. LLM unlearning aims at removing undesired data influences and associated model capabilities without compromising utility out of the scope of unlearning. While interest in studying LLM unlearning is growing,the impact of the optimizer choice for LLM unlearning remains under-explored. In this work, we shed light on the significance of optimizer selection in LLM unlearning for the first time, establishing a clear connection between {second-order optimization} and influence unlearning (a classical approach using influence functions to update the model for data influence removal). This insight propels us to develop a second-order unlearning framework, termed SOUL, built upon the second-order clipped stochastic optimization (Sophia)-based LLM training method. SOUL extends the static, one-shot model update using influence unlearning to a dynamic, iterative unlearning process. Our extensive experiments show that SOUL consistently outperforms conventional first-order methods across various unlearning tasks, models, and metrics, suggesting the promise of second-order optimization in providing a scalable and easily implementable solution for LLM unlearning.

翻译：大型语言模型（LLM）凸显了建立有效遗忘机制的必要性，以遵守数据法规和伦理AI实践。LLM遗忘旨在移除不期望的数据影响及其关联的模型能力，同时不损害遗忘范围之外的模型实用性。尽管对LLM遗忘的研究兴趣日益增长，但优化器选择对LLM遗忘的影响仍未得到充分探索。在本工作中，我们首次阐明了优化器选择在LLM遗忘中的重要性，明确建立了{二阶优化}与影响遗忘（一种利用影响函数更新模型以消除数据影响的经典方法）之间的联系。这一洞察促使我们开发了一种基于二阶裁剪随机优化（Sophia）的LLM训练方法的二阶遗忘框架，称为SOUL。SOUL将基于影响遗忘的静态、一次性模型更新扩展为动态、迭代的遗忘过程。我们的广泛实验表明，SOUL在多种遗忘任务、模型和评估指标上始终优于传统的一阶方法，这表明二阶优化在提供可扩展且易于实现的LLM遗忘解决方案方面具有潜力。