CNNs have become one of the most commonly used computational tool in the past two decades. One of the primary downsides of CNNs is that they work as a ``black box", where the user cannot necessarily know how the image data are analyzed, and therefore needs to rely on empirical evaluation to test the efficacy of a trained CNN. This can lead to hidden biases that affect the performance evaluation of neural networks, but are difficult to identify. Here we discuss examples of such hidden biases in common and widely used benchmark datasets, and propose techniques for identifying dataset biases that can affect the standard performance evaluation metrics. One effective approach to identify dataset bias is to perform image classification by using merely blank background parts of the original images. However, in some situations a blank background in the images is not available, making it more difficult to separate foreground or contextual information from the bias. To overcome this, we propose a method to identify dataset bias without the need to crop background information from the images. That method is based on applying several image transforms to the original images, including Fourier transform, wavelet transforms, median filter, and their combinations. These transforms were applied to recover background bias information that CNNs use to classify images. This transformations affect the contextual visual information in a different manner than it affects the systemic background bias. Therefore, the method can distinguish between contextual information and the bias, and alert on the presence of background bias even without the need to separate sub-images parts from the blank background of the original images. Code used in the experiments is publicly available.

翻译：在过去的二十年中，卷积神经网络已成为最常用的计算工具之一。卷积神经网络的一个主要缺点在于其如同一个“黑箱”，用户未必能知晓图像数据是如何被分析的，因此需要依赖经验性评估来测试已训练卷积神经网络的效能。这可能导致影响神经网络性能评估的隐藏偏差，且此类偏差难以识别。本文讨论了常见且广泛使用的基准数据集中此类隐藏偏差的实例，并提出了识别可能影响标准性能评估指标的数据集偏差的技术。识别数据集偏差的一种有效方法是仅使用原始图像的空白背景部分进行图像分类。然而，在某些情况下，图像中并无空白背景可用，这使得将前景或上下文信息与偏差分离变得更加困难。为克服此问题，我们提出了一种无需从图像中裁剪背景信息即可识别数据集偏差的方法。该方法基于对原始图像施加多种图像变换，包括傅里叶变换、小波变换、中值滤波器及其组合。应用这些变换旨在恢复卷积神经网络用于分类图像的背景偏差信息。这些变换以不同于影响系统性背景偏差的方式影响上下文视觉信息。因此，该方法能够区分上下文信息与偏差，并在无需从原始图像的空白背景中分离子图像部分的情况下，警示背景偏差的存在。实验中使用的代码已公开提供。