A standard practice in developing image recognition models is to train a model on a specific image resolution and then deploy it. However, in real-world inference, models often encounter images different from the training sets in resolution and/or subject to natural variations such as weather changes, noise types and compression artifacts. While traditional solutions involve training multiple models for different resolutions or input variations, these methods are computationally expensive and thus do not scale in practice. To this end, we propose a novel neural network model, parallel-structured and all-component Fourier neural operator (PAC-FNO), that addresses the problem. Unlike conventional feed-forward neural networks, PAC-FNO operates in the frequency domain, allowing it to handle images of varying resolutions within a single model. We also propose a two-stage algorithm for training PAC-FNO with a minimal modification to the original, downstream model. Moreover, the proposed PAC-FNO is ready to work with existing image recognition models. Extensively evaluating methods with seven image recognition benchmarks, we show that the proposed PAC-FNO improves the performance of existing baseline models on images with various resolutions by up to 77.1% and various types of natural variations in the images at inference.

翻译：在开发图像识别模型的标准做法中，通常针对特定图像分辨率训练模型后进行部署。然而，在真实推理场景中，模型常遇到与训练集分辨率不同，或受天气变化、噪声类型和压缩伪影等自然变异影响的图像。传统解决方案需要针对不同分辨率或输入变异训练多个模型，但这些方法计算成本高昂，在实际应用中缺乏可扩展性。为此，我们提出一种新型神经网络模型——并行结构化全分量傅里叶神经算子（PAC-FNO）来解决该问题。与常规前馈神经网络不同，PAC-FNO在频域中运行，使其能在单一模型中处理不同分辨率的图像。我们还提出一种两阶段算法，以最小化对原始下游模型的修改来训练PAC-FNO。此外，所提出的PAC-FNO可直接与现有图像识别模型协同工作。通过七个图像识别基准的全面评估，我们证明所提出的PAC-FNO能将现有基线模型对不同分辨率图像的识别性能提升高达77.1%，并能有效应对推理时图像中的各类自然变异。