Paper weakness identification using single-agent or multi-agent LLMs has attracted increasing attention, yet existing approaches exhibit key limitations. Many multi-agent systems simulate human roles at a surface level, missing the underlying criteria that lead experts to assess complementary intellectual aspects of a paper. Moreover, prior methods implicitly assume identified weaknesses are valid, ignoring reviewer bias, misunderstanding, and the critical role of author rebuttals in validating review quality. Finally, most systems output unranked weakness lists, rather than prioritizing the most consequential issues for users. In this work, we propose DIAGPaper, a novel multi-agent framework that addresses these challenges through three tightly integrated modules. The customizer module simulates human-defined review criteria and instantiates multiple reviewer agents with criterion-specific expertise. The rebuttal module introduces author agents that engage in structured debate with reviewer agents to validate and refine proposed weaknesses. The prioritizer module learns from large-scale human review practices to assess the severity of validated weaknesses and surfaces the top-K severest ones to users. Experiments on two benchmarks, AAAR and ReviewCritique, demonstrate that DIAGPaper substantially outperforms existing methods by producing more valid and more paper-specific weaknesses, while presenting them in a user-oriented, prioritized manner.

翻译：利用单智能体或多智能体大语言模型进行论文缺陷识别已受到越来越多的关注，但现有方法存在关键局限。许多多智能体系统仅表面模拟人类角色，未能捕捉导致专家评估论文互补性智力层面的深层标准。此外，先前方法隐含假设识别出的缺陷是有效的，忽视了审稿人偏见、误解以及作者反驳在验证评审质量中的关键作用。最后，多数系统输出未经排序的缺陷列表，而非为用户优先呈现最具影响的问题。本研究提出DIAGPaper——一个通过三个紧密耦合的模块应对上述挑战的新型多智能体框架。定制器模块模拟人类定义的评审标准，并实例化多个具备特定标准专业知识的审稿人智能体。反驳模块引入作者智能体，与审稿人智能体进行结构化辩论，以验证并精炼提出的缺陷。优先级排序模块从大规模人工评审实践中学习，评估已验证缺陷的严重程度，并向用户呈现最严重的K个缺陷。在AAAR和ReviewCritique两个基准上的实验表明，DIAGPaper显著优于现有方法，能生成更有效且更具论文针对性的缺陷，并以用户导向的优先级排序方式呈现。