Identifying the strengths and limitations of a research paper is a core component of any literature review. However, traditional summaries reflect only the authors' self-presented perspective. Analyzing how other researchers discuss and cite the paper can offer a deeper, more practical understanding of its contributions and shortcomings. In this research, we introduce SECite, a novel approach for evaluating scholarly impact through sentiment analysis of citation contexts. We develop a semi-automated pipeline to extract citations referencing nine research papers and apply advanced natural language processing (NLP) techniques with unsupervised machine learning to classify these citation statements as positive or negative. Beyond sentiment classification, we use generative AI to produce sentiment-specific summaries that capture the strengths and limitations of each target paper, derived both from clustered citation groups and from the full text. Our findings reveal meaningful patterns in how the academic community perceives these works, highlighting areas of alignment and divergence between external citation feedback and the authors' own presentation. By integrating citation sentiment analysis with LLM-based summarization, this study provides a comprehensive framework for assessing scholarly contributions.
翻译:识别研究论文的优势与局限性是任何文献综述的核心组成部分。然而,传统摘要仅反映作者自我陈述的视角。分析其他研究者如何讨论和引用该论文,能够对其贡献与不足提供更深入、更实际的理解。在本研究中,我们提出了SECite,一种通过引用上下文情感分析来评估学术影响力的新方法。我们开发了一个半自动化流程,用于提取引用九篇研究论文的引文,并应用先进的自然语言处理技术和无监督机器学习,将这些引用陈述分类为正面或负面。除了情感分类,我们还利用生成式人工智能生成针对特定情感的摘要,这些摘要从聚类引用组和全文提取,捕捉了每篇目标论文的优势与局限性。我们的研究结果揭示了学术界如何看待这些作品的有意义模式,凸显了外部引用反馈与作者自我陈述之间的一致与分歧之处。通过将引用情感分析与基于LLM的总结相结合,本研究为评估学术贡献提供了一个全面的框架。