A standard practice in developing image recognition models is to train a model on a specific image resolution and then deploy it. However, in real-world inference, models often encounter images different from the training sets in resolution and/or subject to natural variations such as weather changes, noise types and compression artifacts. While traditional solutions involve training multiple models for different resolutions or input variations, these methods are computationally expensive and thus do not scale in practice. To this end, we propose a novel neural network model, parallel-structured and all-component Fourier neural operator (PAC-FNO), that addresses the problem. Unlike conventional feed-forward neural networks, PAC-FNO operates in the frequency domain, allowing it to handle images of varying resolutions within a single model. We also propose a two-stage algorithm for training PAC-FNO with a minimal modification to the original, downstream model. Moreover, the proposed PAC-FNO is ready to work with existing image recognition models. Extensively evaluating methods with seven image recognition benchmarks, we show that the proposed PAC-FNO improves the performance of existing baseline models on images with various resolutions by up to 77.1% and various types of natural variations in the images at inference.

翻译：在开发图像识别模型时，一种标准做法是针对特定图像分辨率训练模型，然后进行部署。然而，在实际推理过程中，模型经常遇到与训练集分辨率不同的图像，或受到天气变化、噪声类型、压缩伪影等自然变异影响的图像。传统解决方案涉及针对不同分辨率或输入变体训练多个模型，但这些方法计算开销大，因此在实际应用中难以扩展。为此，我们提出了一种新型神经网络模型——并行结构全分量傅里叶神经算子（PAC-FNO）来解决这一问题。与常规的前馈神经网络不同，PAC-FNO在频域中运行，使其能够在单一模型中处理不同分辨率的图像。我们还提出了一种两阶段算法，通过最小化对原始下游模型的修改来训练PAC-FNO。此外，所提出的PAC-FNO可随时与现有图像识别模型配合使用。通过在七个图像识别基准上进行广泛评估，我们证明，所提出的PAC-FNO能将现有基线模型在不同分辨率图像上的性能提升高达77.1%，并能处理推理过程中图像的各种自然变异类型。