Recently, MLP-based vision backbones have achieved promising performance in several visual recognition tasks. However, the existing MLP-based methods directly aggregate tokens with static weights, leaving the adaptability to different images untouched. Moreover, Recent research demonstrates that MLP-Transformer is great at creating long-range dependencies but ineffective at catching high frequencies that primarily transmit local information, which prevents it from applying to the downstream dense prediction tasks, such as semantic segmentation. To address these challenges, we propose a content-adaptive yet computationally efficient structure, dubbed Dynamic Spectrum Mixer (DSM). The DSM represents token interactions in the frequency domain by employing the Discrete Cosine Transform, which can learn long-term spatial dependencies with log-linear complexity. Furthermore, a dynamic spectrum weight generation layer is proposed as the spectrum bands selector, which could emphasize the informative frequency bands while diminishing others. To this end, the technique can efficiently learn detailed features from visual input that contains both high- and low-frequency information. Extensive experiments show that DSM is a powerful and adaptable backbone for a range of visual recognition tasks. Particularly, DSM outperforms previous transformer-based and MLP-based models, on image classification, object detection, and semantic segmentation tasks, such as 83.8 \% top-1 accuracy on ImageNet, and 49.9 \% mIoU on ADE20K.

翻译：最近，基于MLP的视觉骨干网络在多项视觉识别任务中取得了显著性能。然而，现有MLP方法直接使用静态权重聚合令牌，缺乏对不同图像的自适应性。此外，近期研究表明，MLP-Transformer擅长建立长距离依赖关系，但在捕获主要传递局部信息的高频分量方面效果不佳，这阻碍了其在语义分割等下游密集预测任务中的应用。为解决这些问题，我们提出一种内容自适应且计算高效的结构——动态频谱混合器（DSM）。DSM通过离散余弦变换在频域中实现令牌交互，能够以对数线性复杂度学习长期空间依赖关系。此外，我们提出动态频谱权重生成层作为频谱波段选择器，可增强信息丰富的频带同时抑制其他频带。通过这种方式，该技术能够从包含高低频信息的视觉输入中高效学习细节特征。大量实验表明，DSM是适用于多种视觉识别任务的强大且可扩展的骨干网络。特别地，DSM在图像分类、目标检测和语义分割任务上优于以往的Transformer和MLP模型，例如在ImageNet上达到83.8%的Top-1准确率，在ADE20K上达到49.9%的mIoU。