Despite the surge in patent applications and emergence of AI drafting tools, systematic evaluation of patent content quality has received limited research attention. To address this gap, We propose to evaluate patents using regulatory compliance, technical coherence, and figure-reference consistency detection modules, and then generate improvement suggestions via an integration module. The framework is validated on a comprehensive dataset comprising 80 human-authored and 80 AI-generated patents from two patent drafting tools. Experimental results show balanced accuracies of 99.74\%, 82.12\%, and 91.2\% respectively across the three detection modules when validated against expert annotations. Additional analysis was conducted to examine defect distributions across patent sections, technical domains, and authoring sources. Section-based analysis indicates that figure-text consistency and technical detail precision require particular attention. Mechanical Engineering and Construction show more claim-specification inconsistencies due to complex technical documentation requirements. AI-generated patents show a significant gap compared to human-authored ones. While human-authored patents primarily contain surface-level errors like typos, AI-generated patents exhibit more structural defects in figure-text alignment and cross-references.
翻译:尽管专利申请数量激增且人工智能起草工具不断涌现,针对专利内容质量的系统性评估研究仍较为有限。为填补这一空白,我们提出通过法规遵从性、技术连贯性及图文一致性检测模块对专利进行评估,并借助集成模块生成改进建议。该框架在一个包含来自两款专利起草工具的80份人工撰写专利与80份AI生成专利的综合数据集上进行了验证。实验结果表明,在与专家标注对比验证时,三个检测模块的平衡准确率分别达到99.74%、82.12%和91.2%。研究进一步通过缺陷分布分析考察了专利章节、技术领域及撰写来源的差异。基于章节的分析表明,图文一致性与技术细节精确度需特别关注;机械工程与建筑领域因复杂技术文档要求,其权利要求书与说明书不一致现象更为突出。AI生成专利与人工撰写专利存在显著差距:人工撰写专利主要包含拼写错误等表层缺陷,而AI生成专利则在图文对应与交叉引用方面表现出更多结构性缺陷。