Past studies have illustrated the prevalence of UI dark patterns, or user interfaces that can lead end-users toward (unknowingly) taking actions that they may not have intended. Such deceptive UI designs can result in adverse effects on end users, such as oversharing personal information or financial loss. While significant research progress has been made toward the development of dark pattern taxonomies, developers and users currently lack guidance to help recognize, avoid, and navigate these often subtle design motifs. However, automated recognition of dark patterns is a challenging task, as the instantiation of a single type of pattern can take many forms, leading to significant variability. In this paper, we take the first step toward understanding the extent to which common UI dark patterns can be automatically recognized in modern software applications. To do this, we introduce AidUI, a novel automated approach that uses computer vision and natural language processing techniques to recognize a set of visual and textual cues in application screenshots that signify the presence of ten unique UI dark patterns, allowing for their detection, classification, and localization. To evaluate our approach, we have constructed ContextDP, the current largest dataset of fully-localized UI dark patterns that spans 175 mobile and 83 web UI screenshots containing 301 dark pattern instances. The results of our evaluation illustrate that \AidUI achieves an overall precision of 0.66, recall of 0.67, F1-score of 0.65 in detecting dark pattern instances, reports few false positives, and is able to localize detected patterns with an IoU score of ~0.84. Furthermore, a significant subset of our studied dark patterns can be detected quite reliably (F1 score of over 0.82), and future research directions may allow for improved detection of additional patterns.

翻译：过往研究已揭示了UI暗模式的普遍性，这类用户界面可能引导终端用户（在不知情的情况下）执行其本无意采取的操作。此类欺骗性UI设计可能对用户造成负面影响，例如过度分享个人信息或导致经济损失。尽管暗模式分类体系的构建已取得显著研究进展，但开发者和用户目前仍缺乏辅助工具来识别、规避及应对这些往往微妙的预设设计模式。然而，暗模式的自动化识别是一项具有挑战性的任务，因为单一模式类型的实例化形式可能千差万别，导致显著的可变性。本文首次探索了在现代软件应用中自动识别常见UI暗模式的可能性。为此，我们提出了AidUI这一创新自动化方法，该方法综合运用计算机视觉与自然语言处理技术，识别应用截图中表征十种特定UI暗模式的视觉与文本线索，从而实现对暗模式的检测、分类与定位。为评估我们的方法，我们构建了ContextDP数据集，这是当前规模最大的全定位UI暗模式数据集，涵盖175张移动端与83张Web端UI截图，包含301个暗模式实例。评估结果表明，AidUI在检测暗模式实例时实现了整体精确率0.66、召回率0.67、F1分数0.65，报告极少误报，且能以约0.84的IoU分数对检测到的模式进行定位。此外，我们研究的暗模式中有显著子集可实现较高可靠性检测（F1分数超过0.82），而未来的研究方向可能为改进其他模式的检测提供可能。