Flaky tests are problematic because they non-deterministically pass or fail for the same software version under test, causing confusion and wasting development effort. While machine learning models have been used to predict flakiness and its root causes, there is much less work on providing support to fix the problem. To address this gap, in this paper, we focus on predicting the type of fix that is required to remove flakiness and then repair the test code on that basis. We do this for a subset of flaky test cases where the root cause of flakiness is in the test case itself and not in the production code. Our key idea is to guide the repair process with additional knowledge about the test's flakiness in the form of its predicted fix category. Thus, we first propose a framework that automatically generates labeled datasets for 13 fix categories and trains models to predict the fix category of a flaky test by analyzing the test code only. Our experimental results using code models and few-shot learning show that we can correctly predict most of the fix categories. To show the usefulness of such fix category labels for automatically repairing flakiness, in addition to informing testers, we augment a Large Language Model (LLM) like GPT with such extra knowledge to ask the LLM for repair suggestions. The results show that our suggested fix category labels significantly enhance the capability of GPT 3.5 Turbo, in generating fixes for flaky tests.

翻译：易碎测试因其在相同软件版本下非确定性通过或失败的特性而引发问题，导致混淆并浪费开发精力。尽管机器学习模型已被用于预测易碎性及其根本原因，但在提供修复支持方面的研究仍较为匮乏。为填补这一空白，本文聚焦于预测消除易碎性所需的修复类型，并据此修复测试代码。我们针对根本原因在于测试用例本身而非生产代码的易碎测试子集展开研究。核心思路是通过预测的修复类别形式，向修复过程注入测试易碎性的额外知识。由此，我们首先提出一种框架，可自动生成涵盖13种修复类别的标注数据集，并通过仅分析测试代码训练模型预测易碎测试的修复类别。基于代码模型和少样本学习的实验结果表明，该方法能正确预测大部分修复类别。为验证此类修复类别标签对自动修复易碎性的实用价值（除告知测试人员外），我们向GPT等大语言模型注入此类额外知识，请求其生成修复建议。实验结果显示，我们建议的修复类别标签显著增强了GPT 3.5 Turbo为易碎测试生成修复方案的能力。