Automated bots now account for roughly half of all web requests, and an increasing number deliberately spoof their identity to either evade detection or to not respect robots.txt. Existing countermeasures are either resource-intensive (JavaScript challenges, CAPTCHAs), cost-prohibitive (commercial solutions), or degrade the user experience. This paper proposes a lightweight, passive approach to bot detection that combines user-agent string analysis with favicon-based heuristics, operating entirely on standard web server logs with no client-side interaction. We evaluate the method on over 4.6 million requests containing 54,945 unique user-agent strings collected from website hosted all around the earth. Our approach detects 67.7% of bot traffic while maintaining a false-positive rate of 3%, outperforming state of the art (less than 20%). This method can serve as a first line of defence, routing only genuinely ambiguous requests to active challenges and preserving the experience of legitimate users.

翻译：自动机器人如今约占所有网络请求的一半，且越来越多机器人故意伪造身份以逃避检测或不遵守robots.txt协议。现有应对措施要么资源消耗大（JavaScript挑战、CAPTCHA验证码），要么成本高昂（商业解决方案），或影响用户体验。本文提出一种轻量级、被动的机器人检测方法，结合用户代理字符串分析与基于网站图标的启发式规则，完全基于标准网络服务器日志运行，无需客户端交互。我们在来自全球各地网站的54645个独特用户代理字符串的超过460万次请求上评估该方法。该方法能检测出67.7%的机器人流量，同时将误报率控制在3%，优于现有技术（不到20%）。该方案可作为第一道防线，仅将真正存疑的请求导向主动验证，从而保障合法用户的体验。