For nearly two decades, CAPTCHAs have been widely used as a means of protection against bots. Throughout the years, as their use grew, techniques to defeat or bypass CAPTCHAs have continued to improve. Meanwhile, CAPTCHAs have also evolved in terms of sophistication and diversity, becoming increasingly difficult to solve for both bots (machines) and humans. Given this long-standing and still-ongoing arms race, it is critical to investigate how long it takes legitimate users to solve modern CAPTCHAs, and how they are perceived by those users. In this work, we explore CAPTCHAs in the wild by evaluating users' solving performance and perceptions of unmodified currently-deployed CAPTCHAs. We obtain this data through manual inspection of popular websites and user studies in which 1,400 participants collectively solved 14,000 CAPTCHAs. Results show significant differences between the most popular types of CAPTCHAs: surprisingly, solving time and user perception are not always correlated. We performed a comparative study to investigate the effect of experimental context -- specifically the difference between solving CAPTCHAs directly versus solving them as part of a more natural task, such as account creation. Whilst there were several potential confounding factors, our results show that experimental context could have an impact on this task, and must be taken into account in future CAPTCHA studies. Finally, we investigate CAPTCHA-induced user task abandonment by analyzing participants who start and do not complete the task.

翻译：近二十年来，验证码（CAPTCHA）被广泛用作抵御自动化程序的防护手段。随着其应用的普及，绕过或破解验证码的技术持续精进。与此同时，验证码在复杂度和多样性方面也在不断演进，使得机器与人类都越来越难以破解。面对这一长期且仍在持续的技术竞赛，探究合法用户解决现代验证码的耗时以及用户对其的感知评价至关重要。本研究通过评估用户对未经修改的当前部署型验证码的解决性能与感知，对实际环境中的验证码进行了探索。我们通过手动检查热门网站以及开展用户研究（共计1400名参与者解决了14000个验证码）来获取相关数据。结果表明，最主流的验证码类型之间存在显著差异：令人意外的是，解决时间与用户感知并非始终相关。我们开展了一项比较研究，以探究实验背景的影响——具体而言，是直接解决验证码与将其作为自然任务（如创建账户）一部分解决之间的差异。尽管存在若干潜在混淆因素，但我们的结果表明，实验背景可能对该任务产生影响，且未来验证码研究中必须考虑此因素。最后，我们通过分析半途而废的参与者数据，对验证码引发的用户任务放弃行为进行了研究。