Recent advances in agentic workflows have enabled the automation of tasks such as professional document generation. However, they primarily focus on textual quality, neglecting visual structure and style, which are crucial for readability and engagement. This gap arises mainly from the absence of suitable reward models to guide agentic workflows toward producing documents with stronger structural and stylistic quality. To address this, we propose DocReward, a document reward model that evaluates documents based on their structure and style. We construct a multi-domain dataset DocPair of 117K paired documents, covering 32 domains and 267 document types, each including a high- and low-professionalism document with identical content but different structure and style. This enables the model to evaluate professionalism comprehensively, and in a textual-quality-agnostic way. DocReward is trained using the Bradley-Terry loss to score documents, penalizing predictions that contradict the annotated ranking. To assess the performance of reward models, we create a test dataset containing document bundles ranked by well-educated human evaluators. Notably, DocReward outperforms GPT-4o and GPT-5 in accuracy by 30.6 and 19.4 percentage points, respectively, demonstrating its superiority over baselines. In an extrinsic evaluation of document generation, DocReward achieves a significantly higher win rate of 60.8%, compared to GPT-5's 37.7% win rate, demonstrating its utility in guiding generation agents toward producing human-preferred documents.

翻译：近年来，智能体工作流的进展已使专业文档生成等任务实现自动化。然而，现有方法主要关注文本质量，忽视了视觉结构与风格——这两者对文档的可读性与吸引力至关重要。这一差距主要源于缺乏合适的奖励模型来引导智能体工作流生成具有更优结构和风格质量的文档。为此，我们提出DocReward，一种基于文档结构与风格进行评估的文档奖励模型。我们构建了一个包含11.7万对文档的多领域数据集DocPair，涵盖32个领域和267种文档类型，每对文档包含内容相同但结构与风格不同的高专业度与低专业度版本。这使得模型能够以与文本质量无关的方式全面评估专业度。DocReward采用Bradley-Terry损失进行训练，通过惩罚与标注排序相矛盾的预测来对文档进行评分。为评估奖励模型的性能，我们创建了一个由受过良好教育的人类评估者排序的文档包测试数据集。值得注意的是，DocReward在准确率上分别超过GPT-4o和GPT-5达30.6和19.4个百分点，显著优于基线模型。在文档生成的外部评估中，DocReward获得了60.8%的显著更高胜率，而GPT-5的胜率为37.7%，这证明了其在引导生成智能体产出更符合人类偏好的文档方面的实用价值。