SourceRank is a scoring system made of 18 metrics that assess the popularity and quality of open-source packages. Despite being used in several recent studies, none has thoroughly analyzed its reliability against evasion attacks aimed at inflating the score of malicious packages, thereby masquerading them as trustworthy. To fill this gap, we first propose a threat model that identifies potential evasion approaches for each metric, including the URL confusion technique, which can affect 5 out of the 18 metrics by leveraging a URL pointing to a legitimate repository potentially unrelated to the malicious package. Furthermore, we study the reliability of SourceRank in the PyPI ecosystem by analyzing the SourceRank distributions of benign and malicious packages in the state-of-the-art MalwareBench dataset, as well as in a real-world dataset of 122,398 packages. Our analysis reveals that, while historical data suggests a clear distinction between benign and malicious packages, the real-world distributions overlap significantly, mainly due to SourceRank's failure to timely reflect package removals. As a result, SourceRank cannot be reliably used to discriminate between benign and malicious packages in real-world scenarios, nor to select benign packages among those available on PyPI. Finally, our analysis reveals that URL confusion represents an emerging attack vector, with its prevalence increasing from 4.2% in MalwareBench to 7.0% in our real-world dataset. Moreover, this technique is often used alongside other evasion techniques and can significantly inflate the SourceRank metrics of malicious packages.

翻译：SourceRank是一个由18项指标构成的评分系统，用于评估开源软件包的流行度与质量。尽管该系统已被多项近期研究采用，但尚未有工作深入分析其在面对旨在提升恶意软件包评分以伪装其可信度的规避攻击时的可靠性。为填补这一空白，我们首先提出一个威胁模型，该模型识别了针对每项指标的潜在规避方法，其中包括URL混淆技术——该技术通过利用指向可能与恶意软件包无关的合法代码仓库的URL，可影响18项指标中的5项。此外，我们通过分析最先进的MalwareBench数据集中良性及恶意软件包的SourceRank分布，并结合对122,398个软件包的真实世界数据集的分析，研究了SourceRank在PyPI生态系统中的可靠性。我们的分析表明，尽管历史数据显示良性包与恶意包之间存在明显区分，但真实世界中的分布存在显著重叠，这主要源于SourceRank未能及时反映软件包的下架情况。因此，在实际场景中，SourceRank无法可靠地区分良性包与恶意包，亦不能用于在PyPI上可用的软件包中筛选良性包。最后，我们的分析揭示URL混淆正成为一种新兴的攻击向量，其流行度从MalwareBench中的4.2%上升至我们真实世界数据集中的7.0%。此外，该技术常与其他规避技术结合使用，并能够显著提升恶意软件包的SourceRank指标得分。