Third-Party Risk Assessment (TPRA) relies on large repositories of cybersecurity compliance questions used to assess external suppliers against standards such as ISO/IEC 27001 and NIST. In practice, not all questions are relevant for a specific supplier and selecting questions for a given assessment context remains a manual and time-consuming task. Existing question retrieval approaches based on lexical or semantic similarity can identify topically related questions, but they often fail to capture the underlying assessment intent, including control domain and evaluation scope. To address this limitation, we investigate whether an explicit semantic label space can improve intent-aware TPRA question selection. In particular, we separate label space discovery from large-scale label assignment. We start by discovering overlapping clusters of semantically similar questions and then exploit LLMs to assign unique labels for each cluster. Second, we propagate labels through k-nearest neighbors (kNN) for a larger-scale question annotation. Question retrieval is finally achieved by similarity measure of the query with respect to the extracted labels instead of the questions themselves. This reduces repeated LLM calls while preserving label consistency. Experimental results show that the proposed semi-supervised framework reduces labeling cost and runtime compared with per-question LLM annotation while maintaining label quality and improving efficiency. Furthermore, label-based retrieval achieves better alignment with cybersecurity control domains and assessment scope than similarity-based retrieval, highlighting the value of semantic labels as an intermediate representation.

翻译：第三方风险评估（TPRA）依赖于大规模网络安全合规问题库，用于依据ISO/IEC 27001和NIST等标准评估外部供应商。实践中，并非所有问题都适用于特定供应商，而为给定评估场景选择问题仍是一项耗时的人工任务。现有基于词汇或语义相似度的问题检索方法虽能识别主题相关的问题，但常无法捕捉潜在的评估意图，包括控制域和评估范围。为解决此局限，我们探究了显式语义标签空间能否改进意图感知的TPRA问题选择。具体而言，我们将标签空间发现与大规模标签分配分离：首先发现语义相似问题的重叠聚类，继而利用大语言模型（LLMs）为每个聚类分配唯一标签；其次，通过k近邻（kNN）传播标签以实现更大规模的问题标注。最终，问题检索通过查询与提取标签而非问题本身的相似度度量来实现。这一做法在保持标签一致性的同时减少了重复的LLM调用。实验结果表明，与逐问题的LLM标注相比，所提出的半监督框架在降低标注成本和运行时开销的同时，维持了标签质量并提升了效率。此外，基于标签的检索在网络安全控制域和评估范围的对齐程度上优于基于相似度的检索，凸显了语义标签作为中间表征的价值。