Binary function similarity, which often relies on learning-based algorithms to identify what functions in a pool are most similar to a given query function, is a sought-after topic in different communities, including machine learning, software engineering, and security. Its importance stems from the impact it has in facilitating several crucial tasks, from reverse engineering and malware analysis to automated vulnerability detection. Whereas recent work cast light around performance on this long-studied problem, the research landscape remains largely lackluster in understanding the resiliency of the state-of-the-art machine learning models against adversarial attacks. As security requires to reason about adversaries, in this work we assess the robustness of such models through a simple yet effective black-box greedy attack, which modifies the topology and the content of the control flow of the attacked functions. We demonstrate that this attack is successful in compromising all the models, achieving average attack success rates of 57.06% and 95.81% depending on the problem settings (targeted and untargeted attacks). Our findings are insightful: top performance on clean data does not necessarily relate to top robustness properties, which explicitly highlights performance-robustness trade-offs one should consider when deploying such models, calling for further research.

翻译：二进制函数相似性通常依赖于基于学习的算法来识别函数池中哪些函数与给定查询函数最为相似，这一课题在机器学习、软件工程和安全等多个领域备受关注。其重要性源于它在促进多项关键任务中的影响，从逆向工程和恶意软件分析到自动化漏洞检测。尽管近期研究揭示了这一长期研究问题上的性能表现，但研究领域在理解最先进机器学习模型对抗对抗性攻击的弹性方面仍然存在显著不足。由于安全领域需要考虑对抗者因素，本研究通过一种简单而有效的黑盒贪婪攻击来评估此类模型的鲁棒性，该攻击会修改被攻击函数的控制流拓扑结构和内容。我们证明该攻击能够成功破坏所有模型，根据问题设置（定向攻击与非定向攻击）实现了57.06%和95.81%的平均攻击成功率。我们的发现具有深刻启示：在干净数据上的优异性能并不必然对应卓越的鲁棒性，这明确凸显了部署此类模型时需要考虑的性能与鲁棒性权衡，并呼吁开展进一步研究。