We propose WaveMix -- a novel neural architecture for computer vision that is resource-efficient yet generalizable and scalable. WaveMix networks achieve comparable or better accuracy than the state-of-the-art convolutional neural networks, vision transformers, and token mixers for several tasks, establishing new benchmarks for segmentation on Cityscapes; and for classification on Places-365, five EMNIST datasets, and iNAT-mini. Remarkably, WaveMix architectures require fewer parameters to achieve these benchmarks compared to the previous state-of-the-art. Moreover, when controlled for the number of parameters, WaveMix requires lesser GPU RAM, which translates to savings in time, cost, and energy. To achieve these gains we used multi-level two-dimensional discrete wavelet transform (2D-DWT) in WaveMix blocks, which has the following advantages: (1) It reorganizes spatial information based on three strong image priors -- scale-invariance, shift-invariance, and sparseness of edges, (2) in a lossless manner without adding parameters, (3) while also reducing the spatial sizes of feature maps, which reduces the memory and time required for forward and backward passes, and (4) expanding the receptive field faster than convolutions do. The whole architecture is a stack of self-similar and resolution-preserving WaveMix blocks, which allows architectural flexibility for various tasks and levels of resource availability. Our code and trained models are publicly available.

翻译：我们提出了WaveMix——一种新型的计算机视觉神经架构，具有资源高效、泛化性强且可扩展的特点。WaveMix网络在多项任务中达到或优于当前最先进的卷积神经网络、视觉Transformer和Token混合器的精度，为Cityscapes上的分割任务、Places-365上的分类任务、五个EMNIST数据集以及iNAT-mini数据集建立了新的基准。值得注意的是，WaveMix架构在实现这些基准时所需的参数量少于先前最先进的模型。此外，在参数量可控的情况下，WaveMix所需的GPU内存更少，从而节省时间、成本和能源。为实现这些增益，我们在WaveMix块中使用了多级二维离散小波变换（2D-DWT），其优势如下：(1) 基于三个强图像先验——尺度不变性、位移不变性和边缘稀疏性——重新组织空间信息，(2) 以无损方式实现且无需增加参数，(3) 同时缩小特征图的空间尺寸，从而减少前向和反向传播所需的内存和时间，以及(4) 比卷积更快地扩展感受野。整个架构由一系列自相似且保持分辨率的WaveMix块堆叠而成，这为不同任务和资源可用性水平提供了架构灵活性。我们的代码和预训练模型已公开提供。