Many recent document embedding models are trained on document-as-image representations, embedding rendered pages as images rather than the underlying source. Meanwhile, existing benchmarks for scientific document retrieval, such as ArXivQA and ViDoRe, treat documents as images of pages, implicitly favoring such representations. In this work, we argue that this paradigm is not well-suited for text-rich multimodal scientific documents, where critical evidence is distributed across structured sources, including text, tables, and figures. To study this setting, we introduce ArXivDoc, a new benchmark constructed from the underlying LaTeX sources of scientific papers. Unlike PDF or image-based representations, LaTeX provides direct access to structured elements (e.g., sections, tables, figures, equations), enabling controlled query construction grounded in specific evidence types. We systematically compare text-only, image-based, and multimodal representations across both single-vector and multi-vector retrieval models. Our results show that: (1) document-as-image representations are consistently suboptimal, especially as document length increases; (2) text-based representations are most effective, even for figure-based queries, by leveraging captions and surrounding context; and (3) interleaved text+image representations outperform document-as-image approaches without requiring specialized training.

翻译：近期众多文档嵌入模型基于“文档即图像”表示进行训练，即将渲染后的页面编码为图像而非底层源文档。与此同时，现有科学文献检索基准（如ArXivQA和ViDoRe）将文档视为页面图像，客观上偏向此类表示方法。本研究指出，这种范式并不适用于文本密集的多模态科学文档——此类文档的关键证据分布于结构化源数据（包括文本、表格和图表）中。为探究该场景，我们基于科学论文的LaTeX源文件构建了新基准ArXivDoc。与PDF或图像表示不同，LaTeX可直接访问结构化元素（如章节、表格、图表、公式），从而支持针对特定证据类型的受控查询构建。我们系统比较了单向量与多向量检索模型在纯文本、纯图像及多模态表示上的表现。结果表明：（1）文档即图像表示始终表现欠佳，且随文档长度增加劣化加剧；（2）基于文本的表示最为有效，即使对于图表类查询，也能通过利用标题和上下文信息取得优势；（3）交错式文本+图像表示无需专门训练即可超越文档即图像方法。