Vulnerabilities in open-source operating systems (OSs) pose substantial security risks to software systems, making their detection crucial. While fuzzing has been an effective vulnerability detection technique in various domains, OS fuzzing (OSF) faces unique challenges due to OS complexity and multi-layered interaction, and has not been comprehensively reviewed. Therefore, this work systematically surveys the state-of-the-art OSF techniques, categorizes them based on the general fuzzing process, and investigates challenges specific to kernel, file system, driver, and hypervisor fuzzing. Finally, future research directions for OSF are discussed.

翻译：开源操作系统中的漏洞对软件系统构成重大安全风险，使其检测至关重要。尽管模糊测试在多个领域已成为有效的漏洞检测技术，但操作系统模糊测试因系统复杂性和多层交互特性面临独特挑战，且尚未得到系统性综述。为此，本文系统梳理了前沿的操作系统模糊测试技术，依据通用模糊测试流程对其进行分类，并深入探究了内核、文件系统、驱动程序和虚拟机监控程序模糊测试的特有挑战。最后，本文讨论了操作系统模糊测试的未来研究方向。