Feature importance methods promise to provide a ranking of features according to importance for a given classification task. A wide range of methods exist but their rankings often disagree and they are inherently difficult to evaluate due to a lack of ground truth beyond synthetic datasets. In this work, we put feature importance methods to the test on real-world data in the domain of cardiology, where we try to distinguish three specific pathologies from healthy subjects based on ECG features comparing to features used in cardiologists' decision rules as ground truth. Some methods generally performed well and others performed poorly, while some methods did well on some but not all of the problems considered.

翻译：特征重要性方法旨在根据特定分类任务的重要性对特征进行排序。现有多种方法，但其排序结果常不一致，且由于缺乏真实世界数据集之外的基准真值，这些方法本质上难以评估。本研究将特征重要性方法应用于心脏病学领域的真实数据，尝试基于心电图特征区分三种特定病理状态与健康受试者，并以心脏病专家决策规则中使用的特征作为基准真值进行比较。部分方法整体表现良好，另一些则表现欠佳，而某些方法仅对部分问题有效，未能覆盖所有检测目标。