Neural network-based techniques for automated program repair are becoming increasingly effective. Despite their success, little is known about why they succeed or fail, and how their way of reasoning about the code to repair compares to human developers. This paper presents the first in-depth study comparing human and neural program repair. In particular, we investigate what parts of the buggy code humans and two state of the art neural repair models focus on. This comparison is enabled by a novel attention-tracking interface for human code editing, based on which we gather a dataset of 98 bug fixing sessions, and on the attention layers of neural repair models. Our results show that the attention of the humans and both neural models often overlaps (0.35 to 0.44 correlation). At the same time, the agreement between humans and models still leaves room for improvement, as evidenced by the higher human-human correlation of 0.56. While the two models either focus mostly on the buggy line or on the surrounding context, the developers adopt a hybrid approach that evolves over time, where 36.8% of the attention is given to the buggy line and the rest to the context. Overall, we find the humans to still be clearly more effective at finding a correct fix, with 67.3% vs. less than 3% correctly predicted patches. The results and data of this study are a first step into a deeper understanding of the internal process of neural program repair, and offer insights inspired by the behavior of human developers on how to further improve neural repair models.

翻译：基于神经网络的自动程序修复技术正日益有效。尽管取得了成功，但人们对其成功或失败的原因，以及它们对修复代码的推理方式与人类开发者的异同仍知之甚少。本文首次深入开展人类与神经程序修复的比较研究。具体而言，我们探究人类与两种最先进的神经修复模型在关注错误代码的哪些部分。这一比较得益于一个用于人类代码编辑的新型注意力追踪界面（基于该界面我们收集了98次错误修复会话的数据集）以及神经修复模型的注意力层。结果表明，人类与两种神经模型的注意力经常重叠（相关性为0.35至0.44）。同时，人类与模型之间的一致性仍有提升空间，人类-人类相关性高达0.56即为佐证。两种模型要么主要关注错误行，要么主要关注周围上下文，而开发者则采取一种随时间演化的混合策略：36.8%的注意力集中于错误行，其余投向上下文。总体而言，我们发现人类在找到正确修复方案方面仍明显更有效，正确预测补丁的比例为67.3%，而模型不足3%。本研究的结果和数据是深入理解神经程序修复内部过程的第一步，并提供了受人类开发者行为启发的见解，以进一步改进神经修复模型。