Recently, deep learning-based techniques have shown promising performance on various tasks related to software engineering. For these learning-based approaches to perform well, obtaining high-quality data is one fundamental and crucial issue. The comment updating task is an emerging software engineering task aiming at automatically updating the corresponding comments based on changes in source code. However, datasets for the comment updating tasks are usually crawled from committed versions in open source software repositories such as GitHub, where there is lack of quality control of comments. In this paper, we focus on cleaning existing comment updating datasets with considering some properties of the comment updating process in software development. We propose a semantic and overlapping-aware approach named CupCleaner (Comment UPdating's CLEANER) to achieve this purpose. Specifically, we calculate a score based on semantics and overlapping information of the code and comments. Based on the distribution of the scores, we filter out the data with low scores in the tail of the distribution to get rid of possible unclean data. We first conducted a human evaluation on the noise data and high-quality data identified by CupCleaner. The results show that the human ratings of the noise data identified by CupCleaner are significantly lower. Then, we applied our data cleaning approach to the training and validation sets of three existing comment updating datasets while keeping the test set unchanged. Our experimental results show that even after filtering out over 30\% of the data using CupCleaner, there is still an improvement in all performance metrics. The experimental results on the cleaned test set also suggest that CupCleaner may provide help for constructing datasets for updating-related tasks.

翻译：近期，基于深度学习的技术在软件工程领域的各项任务中展现出令人瞩目的性能。为确保这些基于学习的方法取得良好效果，获取高质量数据是一个基础且关键的问题。注释更新任务是一项新兴的软件工程任务，旨在根据源代码的变化自动更新相应的注释。然而，注释更新任务的数据集通常从GitHub等开源软件仓库中已提交的版本中爬取，这些仓库中的注释缺乏质量控制。本文聚焦于清洁现有注释更新数据集，同时考虑软件开发过程中注释更新过程的若干特性。我们提出一种名为CupCleaner（Comment UPdating's CLEANER）的语义与重叠感知方法以达到此目的。具体而言，我们基于代码与注释的语义及重叠信息计算一个分数。根据分数分布，我们过滤掉分布尾部的低分数据，以消除可能的不清洁数据。我们首先对CupCleaner识别的噪声数据与高质量数据进行了人工评估。结果表明，CupCleaner识别出的噪声数据的人工评分显著较低。随后，我们将数据清洗方法应用于三个现有注释更新数据集的训练集与验证集，同时保持测试集不变。实验结果显示，即使使用CupCleaner过滤掉超过30%的数据，所有性能指标仍有所提升。在清洗后测试集上的实验结果也表明，CupCleaner可能为构建与更新相关任务的数据集提供帮助。