Large Language Models (LLMs) are increasingly integrated into software applications. Downstream application developers often access LLMs through APIs provided as a service. However, LLM APIs are often updated silently and scheduled to be deprecated, forcing users to continuously adapt to evolving models. This can cause performance regression and affect prompt design choices, as evidenced by our case study on toxicity detection. Based on our case study, we emphasize the need for and re-examine the concept of regression testing for evolving LLM APIs. We argue that regression testing LLMs requires fundamental changes to traditional testing approaches, due to different correctness notions, prompting brittleness, and non-determinism in LLM APIs.

翻译：大型语言模型（LLM）正日益集成到软件应用中。下游应用开发者通常通过作为服务提供的API访问LLM。然而，LLM API经常静默更新并按计划逐步淘汰，迫使用户不断适应变化的模型。这可能导致性能退化并影响提示词设计选择，正如我们在毒性检测案例研究中证明的那样。基于我们的案例研究，我们强调需要对进化中的LLM API进行回归测试，并重新审视这一概念。我们认为，对LLM进行回归测试需要对传统测试方法进行根本性改变，原因包括：正确性概念不同、提示词的脆弱性以及LLM API的非确定性。