Understanding and controlling biasing effects in neural networks is crucial for ensuring accurate and fair model performance. In the context of classification problems, we provide a theoretical analysis demonstrating that the structure of a deep neural network (DNN) can condition the model to assign all predictions to the same class, even before the beginning of training, and in the absence of explicit biases. We prove that, besides dataset properties, the presence of this phenomenon, which we call \textit{Initial Guessing Bias} (IGB), is influenced by model choices including dataset preprocessing methods, and architectural decisions, such as activation functions, max-pooling layers, and network depth. Our analysis of IGB provides information for architecture selection and model initialization. We also highlight theoretical consequences, such as the breakdown of node-permutation symmetry, the violation of self-averaging and the non-trivial effects that depth has on the phenomenon.

翻译：理解并控制神经网络中的偏差效应对于确保模型准确性和公平性至关重要。针对分类问题，我们通过理论分析证明，深度神经网络的结构即使在训练开始之前且缺乏显式偏差的情况下，也可能导致模型将所有预测结果归为同一类别。我们证明，除数据集特性外，这一被称为“初始猜测偏差”（IGB）现象的出现受到模型选择的影响，包括数据预处理方法以及架构决策（如激活函数、最大池化层和网络深度）。对IGB的分析为架构选择与模型初始化提供了参考依据。我们还揭示了理论上的影响，例如节点置换对称性的破缺、自平均特性的失效，以及深度对该现象产生的非平凡效应。