As quantum computing hardware rapidly advances, objectively evaluating the capabilities and error rates of new processors remains a critical challenge for the field. A clear and realistic understanding of current quantum performance is essential to guide research priorities and drive meaningful progress. In this work, we apply and extend a protocol-based benchmarking methodology (Meirom, Mor, Weinstein Arxiv 2505.12441) that utilizes well-defined quantumness thresholds. By evaluating performance at protocol level rather then the gate level, this approach provides a transparent and intuitive assessment of whether specific quantum processors, or isolated sub-chips within them, can demonstrate a practical quantum advantage. To illustrate the utility of this method, we compare two generations of IBM quantum computers: the older Eagle architecture and the newer Heron architecture. Our findings reveal the genuine operational strengths and limitations of these devices, demonstrating substantial performance improvements in the newer Heron generation. This work was made possible by IBM Quantum policies which enable independent and objective assessment on their quantum computers and sub-chips. We strongly encourage other companies to emulate the independent qubit availability and the fair pricing which allow researchers to perform such assessments.

翻译：随着量子计算硬件的快速发展，客观评估新处理器的能力和错误率仍然是该领域面临的关键挑战。清晰地、现实地理解当前的量子计算性能对于指导研究优先级和推动有意义的进步至关重要。在这项工作中，我们应用并扩展了一种基于协议的基准测试方法（Meirom, Mor, Weinstein Arxiv 2505.12441），该方法利用了定义明确的量子性阈值。通过在协议层面而非门层面评估性能，这种方法提供了一种透明且直观的评估标准，以判断特定的量子处理器或其中孤立的子芯片是否能展示出实际的量子优势。为了说明该方法的实用性，我们比较了两代 IBM 量子计算机：较旧的 Eagle 架构和较新的 Heron 架构。我们的研究结果揭示了这些设备的真实运行优势和局限性，展示了较新 Heron 一代在性能上的显著提升。这项工作的实现得益于 IBM Quantum 的相关政策，这些政策允许对其量子计算机和子芯片进行独立且客观的评估。我们强烈鼓励其他公司效仿，提供独立的量子比特访问和公平的定价，从而使研究人员能够进行此类评估。