Open-source LLM agent ecosystems are growing rapidly, yet the security of community-contributed skills - modular tool definitions that extend agent capabilities - remains largely unvetted. The gap we fill: existing scanners operate at the code layer and are structurally blind to instruction-layer and multi-agent risk - natural-language directives that hijack an agent, exfiltrate data through encoded side channels, or chain harm across pipelines - so what is needed is a semantic, multi-dimensional vetting system rather than another signature matcher. We present SKILLVETBENCH, a live public leaderboard on Hugging Face that uses an LLM-as-Judge to vet agent skills. What is new: SARS (Skill Agentic Risk Score), a five-dimensional agentic-risk metric with a principled weighted formula for instruction-following systems. What is integrated: full CVSS v4.0 vector decomposition and a ClawHub dual-view that places our LLM-generated review beside the official marketplace verdict. What is demonstrated: drawing on our companion benchmark paper [ 1], the LLM-as-Judge stage achieves zero false negatives across 78 confirmed-malicious skills and zero false positives across 22 benign controls, while the best static baseline (SKILLSIEVE) still misses 15%; for instruction-layer categories such as Prompt Injection and Memory Poisoning, conventional tools miss between 89% and 100% of threats (e.g., CODEBERT detects none of nine memory-poisoning skills). Detection rates vary from 35% to 95% across four LLM evaluators, motivating ensemble scoring in production deployments.

翻译：开源大语言模型智能体生态系统发展迅猛，但社区贡献的技能——即扩展智能体能力的模块化工具定义——其安全性尚未得到充分验证。现有漏洞扫描工具仅在代码层面运作，在指令层面和多智能体风险层面存在结构性盲区，无法识别诸如劫持智能体的自然语言指令、通过编码侧信道窃取数据、或跨流水线连锁危害等威胁。因此，亟需一种基于语义理解的多维度审核系统，而非传统的特征匹配工具。我们提出SkillVetBench，这是一个部署在Hugging Face上的实时公开排行榜，采用大语言模型作为裁判对智能体技能进行审核。其创新点包括：SARS（技能智能体风险评分）——针对指令遵循系统设计的五维智能体风险度量体系，配备严谨的加权计算公式；完整集成的CVSS v4.0向量分解机制；以及ClawHub双视图功能，可将LLM生成的审核意见与官方市场裁定结果进行对照分析。实验表明：基于我们配套基准论文[1]的方法，LLM裁判阶段在78个确认恶意技能上实现零假阴性，在22个良性控制样本上实现零假阳性，而最佳静态基线方法（SKILLSIEVE）仍遗漏15%的恶意案例；在提示注入和记忆投毒等指令层面攻击类别中，传统工具漏检率高达89%-100%（例如CodeBERT未能检测出9个记忆投毒技能中的任何一个）。四个LLM评估器的检测率在35%至95%之间波动，这促使我们在实际部署中采用集成评分策略。