This paper analyzes a popular loss function used in machine learning called the log-cosh loss function. A number of papers have been published using this loss function but, to date, no statistical analysis has been presented in the literature. In this paper, we present the distribution function from which the log-cosh loss arises. We compare it to a similar distribution, called the Cauchy distribution, and carry out various statistical procedures that characterize its properties. In particular, we examine its associated pdf, cdf, likelihood function and Fisher information. Side-by-side we consider the Cauchy and Cosh distributions as well as the MLE of the location parameter with asymptotic bias, asymptotic variance, and confidence intervals. We also provide a comparison of robust estimators from several other loss functions, including the Huber loss function and the rank dispersion function. Further, we examine the use of the log-cosh function for quantile regression. In particular, we identify a quantile distribution function from which a maximum likelihood estimator for quantile regression can be derived. Finally, we compare a quantile M-estimator based on log-cosh with robust monotonicity against another approach to quantile regression based on convolutional smoothing.

翻译：本文分析了机器学习中一种名为log-cosh损失函数的常用损失函数。尽管已有大量论文使用该损失函数，但截至目前，文献中尚未给出其统计分析。本文提出了生成log-cosh损失的分布函数，将其与类似的柯西分布进行比较，并通过多种统计过程刻画其性质。具体而言，我们考察了其相关的概率密度函数、累积分布函数、似然函数和Fisher信息量。我们并列考虑了柯西分布和Cosh分布，以及位置参数的极大似然估计，包括渐近偏差、渐近方差和置信区间。我们还提供了来自其他几种损失函数（包括Huber损失函数和秩散度函数）的稳健估计量比较。此外，我们研究了log-cosh函数在分位数回归中的应用，特别识别出一种分位数分布函数，由此可推导出分位数回归的极大似然估计量。最后，我们比较了基于log-cosh的具有稳健单调性的分位数M估计量与其他基于卷积平滑的分位数回归方法。