Neural code summarization leverages deep learning models to automatically generate brief natural language summaries of code snippets. The development of Transformer models has led to extensive use of attention during model design. While existing work has primarily and almost exclusively focused on static properties of source code and related structural representations like the Abstract Syntax Tree (AST), few studies have considered human attention, that is, where programmers focus while examining and comprehending code. In this paper, we develop a method for incorporating human attention into machine attention to enhance neural code summarization. To facilitate this incorporation and vindicate this hypothesis, we introduce EyeTrans, which consists of three steps: (1) we conduct an extensive eye-tracking human study to collect and pre-analyze data for model training, (2) we devise a data-centric approach to integrate human attention with machine attention in the Transformer architecture, and (3) we conduct comprehensive experiments on two code summarization tasks to demonstrate the effectiveness of incorporating human attention into Transformers. Integrating human attention leads to an improvement of up to 29.91% in Functional Summarization and up to 6.39% in General Code Summarization performance, demonstrating the substantial benefits of this combination. We further explore performance in terms of robustness and efficiency by creating challenging summarization scenarios in which EyeTrans exhibits interesting properties. We also visualize the attention map to depict the simplifying effect of machine attention in the Transformer by incorporating human attention. This work has the potential to propel AI research in software engineering by introducing more human-centered approaches and data.

翻译：摘要：神经代码摘要利用深度学习模型自动生成代码片段的简洁自然语言摘要。Transformer模型的发展使得注意力机制在模型设计中得到广泛应用。现有研究主要且几乎 exclusively 集中于源代码的静态属性及抽象语法树等相关结构表征，却鲜有研究考虑人类注意力——即程序员在阅读和理解代码时的关注点。本文提出一种将人类注意力融入机器注意力的方法以增强神经代码摘要。为促成这种融合并验证该假设，我们引入EyeTrans，其包含三个步骤：（1）开展大规模眼动追踪人类研究，收集并预分析用于模型训练的数据；（2）设计以数据为中心的方法，将人类注意力与Transformer架构中的机器注意力相整合；（3）在两个代码摘要任务上进行全面实验，证明将人类注意力融入Transformer的有效性。融合人类注意力使功能摘要性能提升高达29.91%，通用代码摘要性能提升达6.39%，充分展示了这种结合的显著优势。我们进一步通过创建具有挑战性的摘要场景来探索鲁棒性和效率方面的性能，EyeTrans在这些场景中展现出有趣的特性。同时可视化注意力图谱，描绘了通过融入人类注意力对Transformer中机器注意力的简化效果。本研究通过引入更以人为中心的方法和数据，有望推动软件工程领域的人工智能研究。