Artificial intelligence (AI) tools are being incorporated into scientific research workflows with the potential to enhance efficiency in tasks such as document analysis, question answering (Q&A), and literature search. However, system outputs are often difficult to verify, lack transparency in their generation and remain prone to errors. Suitable benchmarks are needed to document and evaluate arising issues. Nevertheless, existing benchmarking approaches are not adequately capturing human-centered criteria such as usability, interpretability, and integration into research workflows. To address this gap, the present work proposes and applies a benchmarking framework combining human-centered and computer-centered metrics to evaluate AI-based Q&A and literature review tools for research use. The findings suggest that Q&A tools can offer valuable overviews and generally accurate summaries; however, they are not always reliable for precise information extraction. Explainable AI (xAI) accuracy was particularly low, meaning highlighted source passages frequently failed to correspond to generated answers. This shifted the burden of validation back onto the researcher. Literature review tools supported exploratory searches but showed low reproducibility, limited transparency regarding chosen sources and databases, and inconsistent source quality, making them unsuitable for systematic reviews. A comparison of these tool groups reveals a similar pattern: while AI tools can enhance efficiency in the early stages of the research workflow and shallow tasks, their outputs still require human verification. The findings underscore the importance of explainability features to enhance transparency, verification efficiency and careful integration of AI tools into researchers' workflows. Further, human-centered evaluation remains an important concern to ensure practical applicability.

翻译：人工智能（AI）工具正被纳入科学研究工作流程，有望在文档分析、问答（Q&A）和文献检索等任务中提升效率。然而，系统输出往往难以验证，其生成过程缺乏透明度，且容易出现错误。需要合适的基准来记录和评估这些新出现的问题。然而，现有的基准方法未能充分捕捉以人为中心的指标，例如可用性、可解释性以及与研究工作流程的整合。为填补这一空白，本研究提出并应用了一个结合以人为中心和以计算机为中心指标的基准评估框架，以评估用于研究用途的基于AI的问答和文献综述工具。研究结果表明，问答工具能够提供有价值的概览和通常准确的摘要；然而，它们在精确信息提取方面并非总是可靠的。可解释AI的准确性尤其低，这意味着高亮的源段落往往与生成的答案不匹配，从而将验证的负担转移回研究人员身上。文献综述工具支持探索性检索，但显示出低可重复性、关于所选来源和数据库的有限透明度以及不一致的来源质量，使其不适合用于系统综述。对这些工具组进行比较揭示了类似的模式：虽然AI工具可以提高研究工作流程早期阶段和浅层任务的效率，但其输出仍然需要人工验证。这些发现强调了可解释性功能对于提高透明度、验证效率以及将AI工具仔细整合到研究人员工作流程中的重要性。此外，以人为中心的评估仍然是确保实际适用性的一个重要关注点。