Just like weights, bias terms are the learnable parameters of many popular machine learning models, including neural networks. Biases are thought to enhance the representational power of neural networks, enabling them to solve a variety of tasks in computer vision. However, we argue that biases can be disregarded for some image-related tasks such as image classification, by considering the intrinsic distribution of images in the input space and desired model properties from first principles. Our findings suggest that zero-bias neural networks can perform comparably to biased networks for practical image classification tasks. We demonstrate that zero-bias neural networks possess a valuable property called scalar (multiplication) invariance. This means that the prediction of the network remains unchanged when the contrast of the input image is altered. We extend scalar invariance to more general cases, enabling formal verification of certain convex regions of the input space. Additionally, we prove that zero-bias neural networks are fair in predicting the zero image. Unlike state-of-the-art models that may exhibit bias toward certain labels, zero-bias networks have uniform belief in all labels. We believe dropping bias terms can be considered as a geometric prior in designing neural network architecture for image classification, which shares the spirit of adapting convolutions as the transnational invariance prior. The robustness and fairness advantages of zero-bias neural networks may also indicate a promising path towards trustworthy and ethical AI.

翻译：与权重类似，偏置项也是包括神经网络在内的众多主流机器学习模型的可学习参数。偏置被认为能增强神经网络的表征能力，使其能够解决计算机视觉中的各类任务。然而，我们论证了对于图像分类等特定图像相关任务，通过考虑输入空间中图像的内在分布以及基于基本原理推导的理想模型属性，偏置可以被舍弃。我们的研究表明，在实际图像分类任务中，零偏置神经网络的性能可与含偏置网络相媲美。我们证明零偏置神经网络具备一种称为标量（乘法）不变性的宝贵性质，即网络对输入图像对比度的改变具有预测不变性。我们将标量不变性推广至更一般的情形，从而实现对输入空间中特定凸区域的严格验证。此外，我们证明了零偏置神经网络在对零图像的预测上具有公平性——与可能对特定标签产生偏置的现有最优模型不同，零偏置网络对所有标签具有统一的置信度。我们认为，舍弃偏置项可被视作图像分类神经网络架构设计中的几何先验，其精神与将卷积作为平移不变性先验的做法一脉相承。零偏置神经网络的鲁棒性与公平性优势，或将为构建可信且合乎伦理的人工智能指明一条前景可期的发展路径。