As LLM-based computer-use agents (CUAs) begin to autonomously interact with real-world interfaces, understanding their vulnerability to manipulative interface designs becomes increasingly critical. We introduce SusBench, an online benchmark for evaluating the susceptibility of CUAs to UI dark patterns, designs that aim to manipulate or deceive users into taking unintentional actions. Drawing nine common dark pattern types from existing taxonomies, we developed a method for constructing believable dark patterns on real-world consumer websites through code injections, and designed 313 evaluation tasks across 55 websites. Our study with 29 participants showed that humans perceived our dark pattern injections to be highly realistic, with the vast majority of participants not noticing that these had been injected by the research team. We evaluated five state-of-the-art CUAs on the benchmark. We found that both human participants and agents are particularly susceptible to the dark patterns of Preselection, Trick Wording, and Hidden Information, while being resilient to other overt dark patterns. Our findings inform the development of more trustworthy CUAs, their use as potential human proxies in evaluating deceptive designs, and the regulation of an online environment increasingly navigated by autonomous agents.

翻译：随着基于LLM的计算机使用代理（CUAs）开始自主与现实世界界面交互，理解其对操纵性界面设计的脆弱性变得日益关键。我们提出了SusBench，一种用于评估CUAs对UI暗模式易感性的在线基准，暗模式指旨在操纵或欺骗用户采取非预期行动的设计。我们从现有分类法中提取了九种常见暗模式类型，开发了一种通过代码注入在真实世界消费网站上构建可信暗模式的方法，并在55个网站上设计了313个评估任务。我们对29名参与者的研究表明，人类认为我们的暗模式注入具有高度真实性，绝大多数参与者未注意到这些是由研究团队注入的。我们在该基准上评估了五种最先进的CUAs。研究发现，人类参与者和代理均对预选、欺骗性措辞和隐藏信息这三类暗模式特别易感，而对其他显性暗模式则表现出较强抵抗力。我们的研究结果为开发更可信的CUAs、将其作为评估欺骗性设计的潜在人类代理替代方案，以及规范日益由自主代理主导的在线环境提供了重要参考。