Blockchain smart contracts have emerged as a transformative force in the digital realm, spawning a diverse range of compelling applications. Since solidity smart contracts across various domains manage trillions of dollars in virtual coins, they become a prime target for attacks. One of the primary challenges is keeping abreast of the latest techniques and tools for developing secure smart contracts and examining those already deployed. In this paper, we seek to address these challenges from four aspects: (1) We begin by examining ten automatic tools, specifically focusing on their methodologies and their ability to identify vulnerabilities in solidity smart contracts. (2) We propose a novel criterion for evaluating these tools, based on the ISO/IEC 25010 standard. (3) To facilitate the evaluation of the selected tools, we construct a benchmark that encompasses two distinct datasets: a collection of 389 labelled smart contracts and a scaled set of 20,000 unique cases from real-world contracts. (4) We provide a comparison of the selected tools, offering insights into their strengths and weaknesses and highlighting areas where further improvements are needed. Through this evaluation, we hope to provide developers and researchers with valuable guidance on selecting and using smart contract analysis tools and contribute to the ongoing efforts to improve the security and reliability of smart contracts.

翻译：区块链智能合约已发展成为数字领域的一股变革性力量，催生了多种引人注目的应用。由于各领域的Solidity智能合约管理着价值数万亿美元的虚拟货币，它们因此成为攻击的主要目标。主要挑战之一在于如何紧跟开发安全智能合约的最新技术与工具，并对已部署的合约进行审查。本文旨在从四个方面应对这些挑战：（1）我们首先考察了十种自动化工具，重点关注其方法论及识别Solidity智能合约漏洞的能力；（2）我们基于ISO/IEC 25010标准提出了一种新颖的工具评估准则；（3）为便于对所选取的工具进行评估，我们构建了一个包含两个不同数据集的基准测试集：一组由389个带标签的智能合约组成，另一组是来自真实世界合约的2万个独特案例的规模化集合；（4）我们对所选工具进行了比较，揭示了各自的优势与不足，并指出了需要进一步改进的领域。通过本次评估，我们希望为开发者和研究人员在选取与使用智能合约分析工具方面提供有价值的指导，并为提升智能合约安全性与可靠性的持续努力做出贡献。