Smart contracts are programs deployed on a blockchain and are immutable once deployed. Reentrancy, one of the most important vulnerabilities in smart contracts, has caused millions of dollars in financial loss. Many reentrancy detection approaches have been proposed. It is necessary to investigate the performance of these approaches to provide useful guidelines for their application. In this work, we conduct a large-scale empirical study on the capability of five well-known or recent reentrancy detection tools such as Mythril and Sailfish. We collect 230,548 verified smart contracts from Etherscan and use detection tools to analyze 139,424 contracts after deduplication, which results in 21,212 contracts with reentrancy issues. Then, we manually examine the defective functions located by the tools in the contracts. From the examination results, we obtain 34 true positive contracts with reentrancy and 21,178 false positive contracts without reentrancy. We also analyze the causes of the true and false positives. Finally, we evaluate the tools based on the two kinds of contracts. The results show that more than 99.8% of the reentrant contracts detected by the tools are false positives with eight types of causes, and the tools can only detect the reentrancy issues caused by call.value(), 58.8% of which can be revealed by the Ethereum's official IDE, Remix. Furthermore, we collect real-world reentrancy attacks reported in the past two years and find that the tools fail to find any issues in the corresponding contracts. Based on the findings, existing works on reentrancy detection appear to have very limited capability, and researchers should turn the rudder to discover and detect new reentrancy patterns except those related to call.value().

翻译：智能合约是部署在区块链上的程序，一旦部署即不可更改。重入是智能合约中最重大的漏洞之一，已造成数百万美元的经济损失。目前已有多种重入检测方法被提出。有必要研究这些方法的性能，为其应用提供实用指南。本文对五种知名或较新的重入检测工具（如Mythril和Sailfish）的能力进行了大规模实证研究。我们从Etherscan收集了230,548个已验证的智能合约，经过去重后使用检测工具分析了139,424个合约，发现21,212个合约存在重入问题。随后，我们手动检查了工具在合约中定位的缺陷函数。检查结果显示，我们获得了34个存在重入的真阳性合约和21,178个不存在重入的假阳性合约。我们还分析了真阳性和假阳性的成因。最后，我们基于这两类合约对工具进行了评估。结果表明，工具检测到的重入合约中超过99.8%是假阳性，原因分为八类；且工具仅能检测由call.value()引起的重入问题，其中58.8%可由以太坊官方IDE Remix揭示。此外，我们收集了过去两年报道的真实重入攻击事件，发现这些工具未能发现对应合约中的任何问题。基于这些发现，现有重入检测研究工作能力似乎非常有限，研究人员应调整方向，去发现和检测除与call.value()相关之外的新重入模式。