In the domain of machine learning algorithms, the significance of the loss function is paramount, especially in supervised learning tasks. It serves as a fundamental pillar that profoundly influences the behavior and efficacy of supervised learning algorithms. Traditional loss functions, while widely used, often struggle to handle noisy and high-dimensional data, impede model interpretability, and lead to slow convergence during training. In this paper, we address the aforementioned constraints by proposing a novel robust, bounded, sparse, and smooth (RoBoSS) loss function for supervised learning. Further, we incorporate the RoBoSS loss function within the framework of support vector machine (SVM) and introduce a new robust algorithm named $\mathcal{L}_{rbss}$-SVM. For the theoretical analysis, the classification-calibrated property and generalization ability are also presented. These investigations are crucial for gaining deeper insights into the performance of the RoBoSS loss function in the classification tasks and its potential to generalize well to unseen data. To empirically demonstrate the effectiveness of the proposed $\mathcal{L}_{rbss}$-SVM, we evaluate it on $88$ real-world UCI and KEEL datasets from diverse domains. Additionally, to exemplify the effectiveness of the proposed $\mathcal{L}_{rbss}$-SVM within the biomedical realm, we evaluated it on two medical datasets: the electroencephalogram (EEG) signal dataset and the breast cancer (BreaKHis) dataset. The numerical results substantiate the superiority of the proposed $\mathcal{L}_{rbss}$-SVM model, both in terms of its remarkable generalization performance and its efficiency in training time.
翻译:在机器学习算法领域,损失函数的重要性尤为突出,尤其是在监督学习任务中。它作为基础支柱,深刻影响着监督学习算法的行为与效能。传统损失函数虽被广泛使用,但常难以处理含噪声的高维数据,阻碍模型的可解释性,并导致训练收敛缓慢。本文针对上述局限性,提出一种新型的鲁棒、有界、稀疏且平滑(RoBoSS)损失函数用于监督学习。进一步地,我们将RoBoSS损失函数融入支持向量机(SVM)框架,并引入一种名为$\mathcal{L}_{rbss}$-SVM的新型鲁棒算法。在理论分析方面,本文还给出了分类校准性质与泛化能力的相关论证。这些研究对于深入理解RoBoSS损失函数在分类任务中的性能及其对未见数据的良好泛化潜力至关重要。为实证证明所提$\mathcal{L}_{rbss}$-SVM的有效性,我们在来自不同领域的88个真实世界UCI与KEEL数据集上进行了评估。此外,为示例说明所提$\mathcal{L}_{rbss}$-SVM在生物医学领域的有效性,我们在两个医学数据集上进行了评估:脑电图(EEG)信号数据集与乳腺癌(BreaKHis)数据集。数值结果证实了所提$\mathcal{L}_{rbss}$-SVM模型在显著泛化性能与训练时间效率方面的优越性。