As quantum computing hardware rapidly advances, objectively evaluating the capabilities and error rates of new processors remains a critical challenge for the field. A clear and realistic understanding of current quantum performance is essential for guiding research priorities and driving meaningful progress. In this work, we apply and extend a protocol-based benchmarking methodology (Meirom, Mor, Weinstein Arxiv 2505.12441) that utilizes well-defined \underline{quantumness} thresholds. By evaluating performance at protocol level rather than the gate level, this approach provides a transparent and intuitive assessment of whether specific quantum processors, or isolated sub-chips within them, can demonstrate a practical quantum advantage. To illustrate the utility of this method, we compare two generations of IBM quantum computers: the older Eagle architecture and the newer Heron architecture. Our findings reveal the genuine operational strengths and limitations of these devices, demonstrating substantial performance improvements in the newer Heron generation. This work was made possible by IBM Quantum policies that enable independent and objective assessment of its quantum computers and sub-chips. We strongly encourage other companies to emulate the independent qubit availability and the fair pricing that allow researchers to perform such assessments.

翻译：随着量子计算硬件的快速发展，客观评估新型处理器的能力及错误率仍是该领域的关键挑战。清晰且真实地理解当前量子性能对于指导研究优先级和推动有意义的进展至关重要。在本研究中，我们应用并扩展了一种基于协议的基准测试方法（Meirom, Mor, Weinstein, Arxiv 2505.12441），该方法利用了明确定义的“量子性”阈值。通过在协议层面而非门层面评估性能，这种方法能够透明且直观地判断特定量子处理器或其内部隔离子芯片能否展现出实际的量子优势。为说明该方法的实用性，我们对比了两代IBM量子计算机：较旧的Eagle架构与较新的Heron架构。我们的研究结果揭示了这些设备的真实操作优势与局限，展示了新Heron一代在性能上的显著提升。这项工作得益于IBM量子政策，该政策实现了对其量子计算机及子芯片的独立客观评估。我们强烈鼓励其他公司效仿这种独立量子比特可用性与公平定价模式，使研究人员能够进行此类评估。