A standard practice in developing image recognition models is to train a model on a specific image resolution and then deploy it. However, in real-world inference, models often encounter images different from the training sets in resolution and/or subject to natural variations such as weather changes, noise types and compression artifacts. While traditional solutions involve training multiple models for different resolutions or input variations, these methods are computationally expensive and thus do not scale in practice. To this end, we propose a novel neural network model, parallel-structured and all-component Fourier neural operator (PAC-FNO), that addresses the problem. Unlike conventional feed-forward neural networks, PAC-FNO operates in the frequency domain, allowing it to handle images of varying resolutions within a single model. We also propose a two-stage algorithm for training PAC-FNO with a minimal modification to the original, downstream model. Moreover, the proposed PAC-FNO is ready to work with existing image recognition models. Extensively evaluating methods with seven image recognition benchmarks, we show that the proposed PAC-FNO improves the performance of existing baseline models on images with various resolutions by up to 77.1% and various types of natural variations in the images at inference.

翻译：在图像识别模型的开发中，标准做法是在特定图像分辨率下训练模型然后部署。然而在实际推理中，模型常会遇到分辨率不同于训练集，或受天气变化、噪声类型、压缩伪影等自然变异影响的图像。传统解决方案针对不同分辨率或输入变异训练多个模型，但这类方法计算开销大，实际应用中难以扩展。为此，我们提出一种新型神经网络模型——并行结构全分量傅里叶神经算子（PAC-FNO）来解决该问题。与传统前馈神经网络不同，PAC-FNO在频域中运行，使其能够在单一模型中处理不同分辨率的图像。我们还提出一种两阶段算法，以最小程度修改原始下游模型的方式训练PAC-FNO。此外，所提出的PAC-FNO可即插即用地与现有图像识别模型协同工作。通过七个图像识别基准的广泛评估表明，所提出的PAC-FNO在推理阶段可将现有基线模型对不同分辨率图像的识别性能提升高达77.1%，同时显著提升其对各类自然变异图像的识别表现。