The surge in multimodal single-cell omics data exposes limitations in traditional, manually defined analysis workflows. AI agents offer a paradigm shift, enabling adaptive planning, executable code generation, traceable decisions, and real-time knowledge fusion. However, the lack of a comprehensive benchmark critically hinders progress. We introduce a novel benchmarking evaluation system to rigorously assess agent capabilities in single-cell omics analysis. This system comprises: a unified platform compatible with diverse agent frameworks and LLMs; multidimensional metrics assessing cognitive program synthesis, collaboration, execution efficiency, bioinformatics knowledge integration, and task completion quality; and 50 diverse real-world single-cell omics analysis tasks spanning multi-omics, species, and sequencing technologies. Our evaluation reveals that Grok-3-beta achieves state-of-the-art performance among tested agent frameworks. Multi-agent frameworks significantly enhance collaboration and execution efficiency over single-agent approaches through specialized role division. Attribution analyses of agent capabilities identify that high-quality code generation is crucial for task success, and self-reflection has the most significant overall impact, followed by retrieval-augmented generation (RAG) and planning. This work highlights persistent challenges in code generation, long-context handling, and context-aware knowledge retrieval, providing a critical empirical foundation and best practices for developing robust AI agents in computational biology.

翻译：多模态单细胞组学数据的激增暴露了传统人工定义分析流程的局限性。AI智能体提供了一种范式转变，能够实现自适应规划、可执行代码生成、可追溯决策和实时知识融合。然而，缺乏全面的基准测试严重阻碍了该领域的发展。我们引入了一种新颖的基准评估系统，用于严格评估智能体在单细胞组学分析中的能力。该系统包含：一个兼容多种智能体框架和LLM的统一平台；评估认知程序合成、协作、执行效率、生物信息学知识整合及任务完成质量的多维指标；以及涵盖多组学、多物种、多测序技术的50项多样化真实世界单细胞组学分析任务。我们的评估表明，Grok-3-beta在所有测试的智能体框架中实现了最先进的性能。多智能体框架通过专业化角色分工，在协作和执行效率方面显著优于单智能体方法。对智能体能力的归因分析表明，高质量的代码生成对任务成功至关重要，而自我反思机制具有最显著的整体影响，其次为检索增强生成（RAG）和规划能力。这项工作揭示了当前在代码生成、长上下文处理和情境感知知识检索方面持续存在的挑战，为计算生物学领域开发鲁棒的AI智能体提供了关键的实证基础和最佳实践指南。