Web traffic has evolved to include both human users and automated agents, ranging from benign web crawlers to adversarial scanners such as those capable of credential stuffing, command injection, and account hijacking at the web scale. The estimated financial costs of these adversarial activities are estimated to exceed tens of billions of dollars in 2023. In this work, we introduce WebGuard, a low-overhead in-application forensics engine, to enable robust identification and monitoring of automated web scanners, and help mitigate the associated security risks. WebGuard focuses on the following design criteria: (i) integration into web applications without any changes to the underlying software components or infrastructure, (ii) minimal communication overhead, (iii) capability for real-time detection, e.g., within hundreds of milliseconds, and (iv) attribution capability to identify new behavioral patterns and detect emerging agent categories. To this end, we have equipped WebGuard with multi-modal behavioral monitoring mechanisms, such as monitoring spatio-temporal data and browser events. We also design supervised and unsupervised learning architectures for real-time detection and offline attribution of human and automated agents, respectively. Information theoretic analysis and empirical evaluations are provided to show that multi-modal data analysis, as opposed to uni-modal analysis which relies solely on mouse movement dynamics, significantly improves time-to-detection and attribution accuracy. Various numerical evaluations using real-world data collected via WebGuard are provided achieving high accuracy in hundreds of milliseconds, with a communication overhead below 10 KB per second.

翻译：网络流量已演变为同时包含人类用户与自动化代理，范围从良性网络爬虫到对抗性扫描器，后者能够实施大规模网络凭证填充、命令注入和账户劫持等攻击。据估计，2023年此类对抗活动造成的经济损失预计超过数百亿美元。本研究提出WebGuard——一种低开销的应用程序内取证引擎，旨在实现对自动化网络扫描器的鲁棒识别与监控，并帮助缓解相关安全风险。WebGuard聚焦于以下设计准则：(i) 无需修改底层软件组件或基础设施即可集成至网络应用程序；(ii) 最小化通信开销；(iii) 支持实时检测（例如数百毫秒内）；(iv) 具备归因能力以识别新行为模式并检测新兴代理类别。为此，我们为WebGuard配备了多模态行为监控机制，例如监控时空数据与浏览器事件。同时设计了监督式与非监督式学习架构，分别用于实时检测人类与自动化代理以及离线行为归因。通过信息论分析与实证评估证明：相较于仅依赖鼠标移动动力学的单模态分析，多模态数据分析能显著提升检测时效与归因准确性。利用WebGuard采集的真实数据进行的多项数值评估表明，该系统可在数百毫秒内实现高精度检测，且每秒通信开销低于10 KB。