Organizational cybersecurity policies are often examined to determine whether they adequately comply standard security controls. This task is difficult because control statements are abstract, whereas policy documents describe governance practices in varied natural language. As a result, policy-based control assessment is time-consuming, difficult to standardize, and often difficult to document in a traceable manner. To address this gap, we present PROPARAG, an audit support approach for evaluating organizational cybersecurity policies against security controls autonomously. For each control, the approach retrieves relevant policy evidence, assesses coverage, identifies missing elements, and generates supporting explanations and recommendations. We evaluate PROPARAG on two real-world organizational policy corpora using 1,007 NIST SP 800-53 controls across both closed-source and open-source large language models (LLMs). The framework achieves F1 scores of 88.54 on OrgA and 82.31 on OrgB. The evaluation also shows that PROPARAG identifies relevant gaps in documented organizational policies and generates grounded recommendations for each identified gap. This research provides foundation for LLM-powered autonomous control-level assessment of organizational cybersecurity policies.

翻译：组织网络安全策略常被审查，以确定其是否充分符合标准安全控制措施。由于控制语句具有抽象性，而策略文档以多样化的自然语言描述治理实践，这一任务极具挑战性。因此，基于策略的控制评估既耗时费力、难以标准化，且往往难以形成可追溯的文档记录。为弥合这一差距，我们提出PROPARAG——一种用于自主评估组织网络安全策略对安全控制措施合规性的审计辅助方法。针对每项控制，该方法可检索相关策略证据、评估覆盖范围、识别缺失要素，并生成支持性解释与改进建议。我们在两个真实组织策略语料库上，采用涵盖闭源与开源大语言模型的1,007项NIST SP 800-53控制措施对PROPARAG进行评估。该框架在OrgA与OrgB上分别取得88.54与82.31的F1分数。评估结果同时表明，PROPARAG能识别文档化组织策略中的相关漏洞，并为每个识别漏洞生成基于证据的改进建议。本研究为基于大语言模型的自主化组织网络安全策略控制级评估奠定了基础。