In supervised machine learning, the choice of loss function implicitly assumes a particular noise distribution over the data. For example, the frequently used mean squared error (MSE) loss assumes a Gaussian noise distribution. The choice of loss function during training and testing affects the performance of artificial neural networks (ANNs). It is known that MSE may yield substandard performance in the presence of outliers. The Cauchy loss function (CLF) assumes a Cauchy noise distribution, and is therefore potentially better suited for data with outliers. This papers aims to determine the extent of robustness and generalisability of the CLF as compared to MSE. CLF and MSE are assessed on a few handcrafted regression problems, and a real-world regression problem with artificially simulated outliers, in the context of ANN training. CLF yielded results that were either comparable to or better than the results yielded by MSE, with a few notable exceptions.
翻译:在监督式机器学习中,损失函数的选择隐含地假设了数据服从特定的噪声分布。例如,常用的均方误差(MSE)损失假设高斯噪声分布。训练和测试过程中损失函数的选择会影响人工神经网络(ANN)的性能。已知MSE在存在异常值时可能产生次优结果。柯西损失函数(CLF)假设柯西噪声分布,因此可能更适合处理包含异常值的数据。本文旨在确定与MSE相比,CLF的鲁棒性和泛化能力程度。在ANN训练背景下,针对若干手工设计的回归问题以及一个包含人工模拟异常值的真实回归问题,对CLF和MSE进行了评估。CLF产生的结果与MSE相当或更优,仅有少数显著例外情况。