Patent descriptions must deliver comprehensive technical disclosure while meeting strict legal standards such as enablement and written description requirements. Although large language models have enabled end-to-end automated patent drafting, existing evaluation approaches fail to assess long-form structural coherence and statutory compliance specific to descriptions. We propose Pat-DEVAL, the first multi-dimensional evaluation framework dedicated to patent description bodies. Leveraging the LLM-as-a-judge paradigm, Pat-DEVAL introduces Chain-of-Legal-Thought (CoLT), a legally-constrained reasoning mechanism that enforces sequential patent-law-specific analysis. Experiments validated by patent expert on our Pap2Pat-EvalGold dataset demonstrate that Pat-DEVAL achieves a Pearson correlation of 0.69, significantly outperforming baseline metrics and existing LLM evaluators. Notably, the framework exhibits a superior correlation of 0.73 in Legal-Professional Compliance, proving that the explicit injection of statutory constraints is essential for capturing nuanced legal validity. By establishing a new standard for ensuring both technical soundness and legal compliance, Pat-DEVAL provides a robust methodological foundation for the practical deployment of automated patent drafting systems.
翻译:专利说明书必须提供全面的技术公开,同时满足严格的法律标准,如可实施性与书面描述要求。尽管大型语言模型已实现端到端的自动化专利撰写,但现有评估方法无法针对说明书特有的长篇结构连贯性与法定合规性进行评估。我们提出了Pat-DEVAL,首个专用于专利说明书主体的多维度评估框架。该框架利用LLM-as-a-judge范式,引入了链式法律思维(CoLT)——一种受法律约束的推理机制,强制执行顺序性的专利法特定分析。在Pap2Pat-EvalGold数据集上经专利专家验证的实验表明,Pat-DEVAL实现了0.69的皮尔逊相关系数,显著优于基线指标与现有LLM评估器。值得注意的是,该框架在法律专业合规性维度上表现出0.73的优异相关性,证明明确注入法定约束对于捕捉细微法律有效性至关重要。通过为保障技术合理性与法律合规性建立新标准,Pat-DEVAL为自动化专利撰写系统的实际部署提供了坚实的方法论基础。