Third-party skills extend LLM agents with powerful capabilities but often handle sensitive credentials in privileged environments, making leakage risks poorly understood. We present the first large-scale empirical study of this problem, analyzing 17,022 skills (sampled from 170,226 on SkillsMP) using static analysis, sandbox testing, and manual inspection. We identify 520 vulnerable skills with 1,708 issues and derive a taxonomy of 10 leakage patterns (4 accidental and 6 adversarial). We find that (1) leakage is fundamentally cross-modal: 76.3% require joint analysis of code and natural language, while 3.1% arise purely from prompt injection; (2) debug logging is the primary vector, with print and console.log causing 73.5% of leaks due to stdout exposure to LLMs; and (3) leaked credentials are both exploitable (89.6% without privileges) and persistent, as forks retain secrets even after upstream fixes. After disclosure, all malicious skills were removed and 91.6% of hardcoded credentials were fixed. We release our dataset, taxonomy, and detection pipeline to support future research.
翻译:第三方技能为LLM代理提供了强大能力,但通常在特权环境中处理敏感凭证,导致泄露风险尚不明确。我们首次对此问题开展大规模实证研究,通过静态分析、沙箱测试与人工审查,对来自SkillsMP的170,226个技能中抽样的17,022个技能进行分析。共识别出520个脆弱技能、1,708个问题,并推导出包含10种泄露模式(4种偶然性、6种对抗性)的分类体系。研究发现:(1)泄露本质上是跨模态的:76.3%需要结合代码与自然语言进行联合分析,3.1%完全由提示注入引发;(2)调试日志是主要传播途径:print和console.log因将标准输出暴露给LLM,导致73.5%的泄露;(3)泄露凭证不仅可被利用(89.6%无需特权),且具有持久性——即使上游修复后,fork分支仍保留秘密。经披露后,所有恶意技能已被删除,91.6%的硬编码凭证得到修复。我们公开了数据集、分类体系及检测管道,以支持后续研究。