Deep neural networks (DNNs) trained with the logistic loss (i.e., the cross entropy loss) have made impressive advancements in various binary classification tasks. However, generalization analysis for binary classification with DNNs and logistic loss remains scarce. The unboundedness of the target function for the logistic loss is the main obstacle to deriving satisfactory generalization bounds. In this paper, we aim to fill this gap by establishing a novel and elegant oracle-type inequality, which enables us to deal with the boundedness restriction of the target function, and using it to derive sharp convergence rates for fully connected ReLU DNN classifiers trained with logistic loss. In particular, we obtain optimal convergence rates (up to log factors) only requiring the H\"older smoothness of the conditional class probability $\eta$ of data. Moreover, we consider a compositional assumption that requires $\eta$ to be the composition of several vector-valued functions of which each component function is either a maximum value function or a H\"older smooth function only depending on a small number of its input variables. Under this assumption, we derive optimal convergence rates (up to log factors) which are independent of the input dimension of data. This result explains why DNN classifiers can perform well in practical high-dimensional classification problems. Besides the novel oracle-type inequality, the sharp convergence rates given in our paper also owe to a tight error bound for approximating the natural logarithm function near zero (where it is unbounded) by ReLU DNNs. In addition, we justify our claims for the optimality of rates by proving corresponding minimax lower bounds. All these results are new in the literature and will deepen our theoretical understanding of classification with DNNs.

翻译：使用逻辑损失（即交叉熵损失）训练的深层神经网络（DNNs）在各类二分类任务中取得了显著进展。然而，关于DNNs与逻辑损失在二分类中的泛化分析仍然较为稀缺。逻辑损失目标函数的无界性是推导满意泛化界的主要障碍。本文旨在通过建立一种新颖而优雅的预言不等式来填补这一空白——该不等式使我们能够处理目标函数的有界性限制——并利用该不等式推导出使用逻辑损失训练的完全连接ReLU DNN分类器的精确收敛速率。特别地，我们仅需数据条件类概率$\eta$的Hölder光滑性，即可获得最优收敛速率（最多相差对数因子）。此外，我们考虑一种组合假设，即$\eta$由多个向量值函数复合而成，其中每个分量函数或为最大值函数，或为仅依赖少量输入变量的Hölder光滑函数。在此假设下，我们推导出与数据输入维度无关的最优收敛速率（最多相差对数因子）。这一结果解释了DNN分类器为何能在高维实际分类问题中表现优异。除新颖的预言不等式外，本文给出的精确收敛速率还归因于ReLU DNN在零点附近（此点函数无界）对自然对数函数的紧致误差逼近界。此外，我们通过证明相应的极小值下界，验证了所得速率的优化性声明。上述结果均为文献中的新贡献，并将深化我们对DNN分类问题的理论认识。