Despite the tremendous advances in machine learning (ML), training with imbalanced data still poses challenges in many real-world applications. Among a series of diverse techniques to solve this problem, sampling algorithms are regarded as an efficient solution. However, the problem is more fundamental, with many works emphasizing the importance of instance hardness. This issue refers to the significance of managing unsafe or potentially noisy instances that are more likely to be misclassified and serve as the root cause of poor classification performance. This paper introduces HardVis, a visual analytics system designed to handle instance hardness mainly in imbalanced classification scenarios. Our proposed system assists users in visually comparing different distributions of data types, selecting types of instances based on local characteristics that will later be affected by the active sampling method, and validating which suggestions from undersampling or oversampling techniques are beneficial for the ML model. Additionally, rather than uniformly undersampling/oversampling a specific class, we allow users to find and sample easy and difficult to classify training instances from all classes. Users can explore subsets of data from different perspectives to decide all those parameters, while HardVis keeps track of their steps and evaluates the model's predictive performance in a test set separately. The end result is a well-balanced data set that boosts the predictive power of the ML model. The efficacy and effectiveness of HardVis are demonstrated with a hypothetical usage scenario and a use case. Finally, we also look at how useful our system is based on feedback we received from ML experts.

翻译：尽管机器学习取得了巨大进步，但在诸多实际应用中，非平衡数据训练仍带来挑战。在解决该问题的系列技术中，采样算法被视为高效解决方案。然而该问题更具根本性，诸多研究强调实例硬度的重要性——即管理容易被误分类、且成为分类性能低下根本原因的不可靠或潜在噪声实例的意义。本文提出HardVis，一个专为处理非平衡分类场景中实例硬度而设计的可视分析系统。该系统辅助用户：直观比较不同数据类型的分布、基于局部特征选择将受主动采样方法影响的实例类型、验证欠采样或过采样技术的哪些建议对机器学习模型有益。此外，区别于对特定类别进行统一采样，用户可发现并选择所有类别中易于/难以分类的训练实例。用户可从多视角探索数据子集以决定所有参数，而HardVis则持续记录操作步骤并独立评估模型在测试集上的预测性能。最终生成的均衡数据集将提升机器学习模型的预测能力。通过假设场景用例与实际应用案例验证了HardVis的有效性与效率，并基于机器学习专家反馈评估了系统的实用性。