State-of-the-art deep CNN face matchers are typically created using extensive training sets of color face images. Our study reveals that such matchers attain virtually identical accuracy when trained on either grayscale or color versions of the training set, even when the evaluation is done using color test images. Furthermore, we demonstrate that shallower models, lacking the capacity to model complex representations, rely more heavily on low-level features such as those associated with color. As a result, they display diminished accuracy when trained with grayscale images. We then consider possible causes for deeper CNN face matchers "not seeing color". Popular web-scraped face datasets actually have 30 to 60% of their identities with one or more grayscale images. We analyze whether this grayscale element in the training set impacts the accuracy achieved, and conclude that it does not. We demonstrate that using only grayscale images for both training and testing achieves accuracy comparable to that achieved using only color images for deeper models. This holds true for both real and synthetic training datasets. HSV color space, which separates chroma and luma information, does not improve the network's learning about color any more than in the RGB color space. We then show that the skin region of an individual's images in a web-scraped training set exhibits significant variation in their mapping to color space. This suggests that color carries limited identity-specific information. We also show that when the first convolution layer is restricted to a single filter, models learn a grayscale conversion filter and pass a grayscale version of the input color image to the next layer. Finally, we demonstrate that leveraging the lower per-image storage for grayscale to increase the number of images in the training set can improve accuracy of the face recognition model.

翻译：当前最先进的深度CNN人脸匹配器通常使用大量彩色人脸图像训练集构建。我们的研究表明，即使使用彩色测试图像进行评估，此类匹配器在灰度或彩色版本训练集上训练时获得的准确率几乎完全相同。此外，我们证明较浅的模型由于缺乏建模复杂表征的能力，更依赖于低级特征（如与颜色相关的特征），因此在灰度图像训练时表现出准确率下降。我们随后探讨了深度CNN人脸匹配器"不感知颜色"的可能原因：流行的网络爬取人脸数据集中，实际上有30%至60%的身份包含一张或多张灰度图像。通过分析训练集中的灰度元素是否影响最终准确率，我们得出结论——这种影响并不存在。实验证明，对于深度模型而言，仅使用灰度图像进行训练和测试所达到的准确率，与仅使用彩色图像训练的结果相当。这一结论在真实数据集和合成训练集上均成立。HSV色彩空间虽分离了色度与亮度信息，但并未比RGB色彩空间更有效地增强网络对颜色的学习能力。进一步研究发现，网络爬取训练集中个体图像的皮肤区域在色彩空间映射上存在显著差异，这表明颜色携带的身份特异性信息有限。我们还发现，当限制第一卷积层为单滤波器时，模型会学习灰度转换滤波器，并将输入彩色图像转换为灰度版本传递至后续层。最后，我们证明利用灰度图像较低的单图存储成本来增加训练集图像数量，能够有效提升人脸识别模型的准确率。