While machine learning (ML) software necessitates effective quality assurance, ML engineers still encounter silent semantic faults, such as imbalanced datasets, that degrade prediction performance without apparent symptoms. These faults are typically detected after expensive training cycles, causing significant resource waste. We propose a data-informed static analysis technique to detect silent semantic faults in ML scripts that use the popular random forest classifier. Our approach extracts ML pipelines into directed acyclic graphs and evaluates them against formalized API contracts to detect structural, data, and hyperparameter faults. Our analysis uses aggregated data properties, enabling fault detection even when datasets are inaccessible due to confidentiality restrictions. We implemented this technique in an open-source tool, dille, and evaluated it on real-world Kaggle notebooks that use the random forest classifier. Our results demonstrate that the tool identifies relevant semantic faults with 91% precision and sub-second runtime overhead, making it suitable for integration into integrated development environments, agentic workflows, and continuous integration pipelines. Our empirical study reveals that 12% to 18% of existing ML notebooks that use the random forest classifier are affected by silent semantic faults, highlighting the immediate practical utility of data-informed static analysis in reducing the burden of ML debugging.

翻译：尽管机器学习软件亟需有效的质量保障，但ML工程师仍会遭遇静默语义故障（如数据集不平衡等），这类故障会在无明显症状的情况下降低预测性能。这些故障通常需要在昂贵的训练周期后才能被发现，造成严重的资源浪费。我们提出一种数据信息静态分析技术，用于检测使用流行随机森林分类器的ML脚本中的静默语义故障。该方法将ML流水线提取为有向无环图，并依据形式化的API契约对其进行评估，以检测结构、数据和超参数故障。分析过程中使用聚合数据属性，即使因保密限制无法访问数据集，仍能实现故障检测。我们将该技术实现为开源工具dille，并在使用随机森林分类器的真实Kaggle笔记本上进行了评估。结果表明，该工具能以91%的精确率和亚秒级运行时开销识别相关语义故障，适合集成到集成开发环境、智能体工作流和持续集成流水线中。我们的实证研究发现，12%至18%使用随机森林分类器的现有ML笔记本受到静默语义故障影响，这凸显了数据信息静态分析在减轻ML调试负担方面的即时效用。