Most enterprise document AI today is a pipeline. Parse, index, retrieve, generate. Each of those stages has been studied to death on its own -- what's still hard is evaluating the system as a whole. We built EnterpriseDocBench to take a swing at it: parsing fidelity, indexing efficiency, retrieval relevance, and generation groundedness, all on the same corpus. The corpus is built from public, permissively licensed documents across six enterprise domains (five represented in the current pilot). We ran three pipelines through it -- BM25, dense embedding, and a hybrid -- all with the same GPT-5 generator. The headline numbers: hybrid retrieval narrowly beats BM25 (nDCG@5 of 0.92 vs. 0.91), and both beat dense embedding (0.83). Hallucination doesn't grow monotonically with document length -- short documents and very long ones both hallucinate more than medium ones (28.1% and 23.8% vs. 9.2%). Cross-stage correlations are very weak: parsing->retrieval r=0.14, parsing->generation r=0.17, retrieval->generation 0.02. If quality were cascading the way most of us assume, those numbers would be much higher; they aren't. Design caveats are real (parsing fixed, generator shared, automated proxy metrics) and we don't oversell the result. One result that genuinely surprised us: factual accuracy on stated claims is 85.5%, but answer completeness averages 0.40. The system is right when it answers -- it just leaves things out. That gap matters more for real deployments than the headline accuracy number does. We also describe three reference architectures (ColPali, ColQwen2, agentic complexity-based routing) which are not yet integrated end-to-end. Framework, metrics, baselines, and collection scripts will be released open-source on acceptance.

翻译：当前大多数企业文档AI采用流水线架构：解析、索引、检索、生成。每个阶段均已得到充分独立研究——真正的难点在于系统级评估。为此我们构建了EnterpriseDocBench，在统一语料库上同步评估解析保真度、索引效率、检索相关性及生成忠实度。该语料库源自六大企业领域（当前试点涵盖五个领域）的公开许可文档。我们基于同一GPT-5生成器，在语料库上测试了三种流水线——BM25、稠密嵌入及混合架构。主要数据如下：混合检索以微弱优势超越BM25（nDCG@5分别为0.92与0.91），两者均显著优于稠密嵌入（0.83）。幻觉率与文档长度呈非线性关系：短文档（28.1%）和超长文档（23.8%）的幻觉率均高于中等长度文档（9.2%）。跨阶段相关性极弱：解析→检索为r=0.14，解析→生成为r=0.17，检索→生成为0.02。若质量按业界普遍假设的级联方式传递，这些数值应显著更高——但事实并非如此。实验设计存在固有局限（固定解析器、共享生成器、自动代理指标），我们并不夸大结论。真正令我们意外的发现是：陈述性事实的准确率达85.5%，但答案完整性均值为0.40。系统在应答时正确，却存在信息遗漏。对实际部署而言，此完整性差距比表层准确率数据更具意义。此外，我们描述了三种参考架构（ColPali、ColQwen2、基于智能体复杂度的路由），但尚未实现端到端集成。框架、指标、基准测试及数据采集脚本将在论文接收后开源发布。