Large Language Models (LLMs) are frequently updated due to data or architecture changes to improve their performance. When updating models, developers often focus on increasing overall performance metrics with less emphasis on being compatible with previous model versions. However, users often build a mental model of the functionality and capabilities of a particular machine learning model they are interacting with. They have to adapt their mental model with every update -- a draining task that can lead to user dissatisfaction. In practice, fine-tuned downstream task adapters rely on pretrained LLM base models. When these base models are updated, these user-facing downstream task models experience instance regression or negative flips -- previously correct instances are now predicted incorrectly. This happens even when the downstream task training procedures remain identical. Our work aims to provide seamless model updates to a user in two ways. First, we provide evaluation metrics for a notion of compatibility to prior model versions, specifically for generative tasks but also applicable for discriminative tasks. We observe regression and inconsistencies between different model versions on a diverse set of tasks and model updates. Second, we propose a training strategy to minimize the number of inconsistencies in model updates, involving training of a compatibility model that can enhance task fine-tuned language models. We reduce negative flips -- instances where a prior model version was correct, but a new model incorrect -- by up to 40% from Llama 1 to Llama 2.

翻译：大型语言模型（LLMs）常因数据或架构变更而更新以提升性能。在更新模型时，开发者通常关注整体性能指标的提升，而较少强调与先前模型版本的兼容性。然而，用户往往会对所交互的特定机器学习模型的功能与能力形成心智模型。每次更新时他们都必须调整自己的心智模型——这是一项令人疲惫的任务，可能导致用户不满。在实践中，经过微调的下游任务适配器依赖于预训练的LLM基础模型。当这些基础模型更新时，这些面向用户的下游任务模型会出现实例回归或负翻转现象——即先前预测正确的实例现在被错误预测。即使下游任务的训练流程保持不变，这种现象依然会发生。本研究旨在通过两种方式为用户提供无缝的模型更新体验。首先，我们提出了一套评估指标，用于量化模型与先前版本的兼容性概念，该指标主要针对生成式任务设计，但也适用于判别式任务。我们在多种任务和模型更新场景中观察到不同模型版本间的回归与不一致现象。其次，我们提出了一种训练策略以最小化模型更新过程中的不一致性，该策略通过训练兼容性模型来增强经过任务微调的语言模型。在从Llama 1到Llama 2的更新过程中，我们将负翻转（即先前版本预测正确而新版本预测错误的实例）减少了高达40%。