Previous work in Neural Loss Function Search (NLFS) has shown a lack of correlation between smaller surrogate functions and large convolutional neural networks with massive regularization. We expand upon this research by revealing another disparity that exists, correlation between different types of image augmentation techniques. We show that different loss functions can perform well on certain image augmentation techniques, while performing poorly on others. We exploit this disparity by performing an evolutionary search on five types of image augmentation techniques in the hopes of finding image augmentation specific loss functions. The best loss functions from each evolution were then taken and transferred to WideResNet-28-10 on CIFAR-10 and CIFAR-100 across each of the five image augmentation techniques. The best from that were then taken and evaluated by fine-tuning EfficientNetV2Small on the CARS, Oxford-Flowers, and Caltech datasets across each of the five image augmentation techniques. Multiple loss functions were found that outperformed cross-entropy across multiple experiments. In the end, we found a single loss function, which we called the inverse bessel logarithm loss, that was able to outperform cross-entropy across the majority of experiments.

翻译：先前在神经损失函数搜索（NLFS）领域的研究发现，较小规模的替代函数与具有大规模正则化的大型卷积神经网络之间存在相关性缺失。我们在此基础上进一步揭示了另一类差异——不同图像增强技术类型之间的相关性。研究表明，特定损失函数在某些图像增强技术中表现优异，但在其他技术中效果欠佳。我们利用这一差异，对五种图像增强技术进行了进化搜索，旨在发现图像增强专用的损失函数。随后，从每次进化中提取最优损失函数，将其迁移至基于CIFAR-10和CIFAR-100数据集、采用五种图像增强技术的WideResNet-28-10网络。进一步地，从上述结果中筛选最优损失函数，通过微调EfficientNetV2Small在CARS、Oxford-Flowers和Caltech数据集上对其进行评估，每个数据集均覆盖五种图像增强技术。实验发现，多个损失函数在各项实验中均优于交叉熵损失。最终，我们提出一种名为逆贝塞尔对数损失的单一损失函数，其在大多数实验中的表现均超越交叉熵损失。